-- Logs begin at Sat 2022-07-02 02:26:01 UTC, end at Sat 2022-07-02 04:03:09 UTC. -- Jul 02 03:19:52.456104 dv-cinder-846842-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 02 03:19:52.467202 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jul 2 03:19:52 2022] *** Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: nodename: dv-cinder-846842-9-iscsi Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: machine: x86_64 Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: clock source: unix Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: pcre jit disabled Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: detected number of CPU cores: 4 Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: current working directory: / Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: detected binary path: /usr/bin/uwsgi-core Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: your processes number limit is 47680 Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: your memory page size is 4096 bytes Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: detected max file descriptor number: 2048 Jul 02 03:19:52.469330 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: lock engine: pthread robust mutexes Jul 02 03:19:52.470128 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: thunder lock: enabled Jul 02 03:19:52.470128 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 02 03:19:52.470128 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 02 03:19:52.514320 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: Python main interpreter initialized at 0x5642a73669c0 Jul 02 03:19:52.514320 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: python threads support enabled Jul 02 03:19:52.514320 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: your server socket listen backlog is limited to 100 connections Jul 02 03:19:52.514320 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: your mercy for graceful operations on workers is 90 seconds Jul 02 03:19:52.514670 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: mapped 403077 bytes (393 KB) for 2 cores Jul 02 03:19:52.514743 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: *** Operational MODE: preforking *** Jul 02 03:19:52.515001 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: *** uWSGI is running in multiple interpreter mode *** Jul 02 03:19:52.515001 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: spawned uWSGI master process (pid: 80343) Jul 02 03:19:52.515210 dv-cinder-846842-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 02 03:19:52.516338 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: spawned uWSGI worker 1 (pid: 80353, cores: 1) Jul 02 03:19:52.516338 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: spawned uWSGI worker 2 (pid: 80354, cores: 1) Jul 02 03:19:52.516338 dv-cinder-846842-9-iscsi devstack@keystone.service[80343]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 02 03:19:57.915934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 02 03:19:57.915934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: from cryptography.utils import int_from_bytes Jul 02 03:19:57.916761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 02 03:19:57.916761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: from cryptography.utils import int_from_bytes Jul 02 03:19:57.920479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 02 03:19:57.920479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: from cryptography.utils import int_from_bytes Jul 02 03:19:57.920991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 02 03:19:57.920991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: from cryptography.utils import int_from_bytes Jul 02 03:19:57.964885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 02 03:19:57.965073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 02 03:19:57.965219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] command line args: [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 02 03:19:57.965511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 02 03:19:57.965642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ================================================================================ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 02 03:19:57.965928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] admin_token = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.966178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config_dir = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.966380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.966515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config_source = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.966716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] debug = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.966882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.967251 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.967438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.967721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.967970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.968168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.968258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 02 03:19:57.968334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_config_append = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.968394 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 02 03:19:57.968565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.968643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] command line args: [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 02 03:19:57.968701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_dir = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.968842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 02 03:19:57.968929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.968995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ================================================================================ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 02 03:19:57.969058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.969253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] admin_token = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.969349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.969513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config_dir = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.969597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.969708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.969873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.969958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] config_source = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] debug = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.970884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971244 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971405 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_config_append = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_dir = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.971940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.972054 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.972185 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.972267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.972488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.972666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.972755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.972828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] publish_errors = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.972942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.973992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.974139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] standard_threads = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.974229 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.974366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.974454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.974604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.974693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.974772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.974854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.974933 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_journal = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.975096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.975198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_json = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.975273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_stderr = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.975425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.975512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_syslog = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.975660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.975883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.976032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.976113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] publish_errors = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.976190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.976268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.976392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.976486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.976620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.976701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.976868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.976949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.977100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.977189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.977333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] standard_threads = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.977415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.977566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.977647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.977807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.977895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.977977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.978048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.978123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_journal = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.978287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.978374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_json = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.978460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_stderr = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.978539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.978617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] use_syslog = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.978749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.978830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 02 03:19:57.978976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.979072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.979220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.979336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.979485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.979568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.979685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.979803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.979953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.980034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.980248 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.980472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.980748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.980847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.980953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.981038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.981245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.981321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.981503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.981659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.981759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.981843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.982010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.982092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.982241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.982327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.982470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.982554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.982702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.982781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.982948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.983033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.983194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.983272 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.983448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.983527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.983677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.983770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.983924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.984001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.984169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.984246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.984401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.984487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.984647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.984724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.984873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.984956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.985105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.985190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.985336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.985462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.985563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.985640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.985792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.985871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.986023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.986107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.986248 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.986338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.986481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.986561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.986727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.986803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.986952 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.987036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.987186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.987259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.987421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.987501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.987656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.987735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.987891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.987971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.988137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.988214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.988366 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.988449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.988598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.988677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.988831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.988907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.989061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.989138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.989290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.989373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.989519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.989622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.989745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.989856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.989992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.990101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.990225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.990333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.990456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.990588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.990684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.990758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.990926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.991015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.991157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.991238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.991407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.991489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.991640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.991758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.991874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.991955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.992140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.992228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.992376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.992461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.992614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.992699 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.992850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.992977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.993115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.993202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.993309 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.993388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.993543 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.993632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.993783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.993912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.994012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.994090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.994244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.994341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.994479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.994557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.994710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.994791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.994943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.995029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.995175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.995281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.995411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.995484 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.995648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.995798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.996031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.996214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.996419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.996503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.996673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.996759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.996912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.996991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.997145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.997223 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.997380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.997517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.997612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.997771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.997851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.997922 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.998053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.998141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.998278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.998357 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.998509 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.998586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.998735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.998891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.998967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.999038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.999189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.999267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.999454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.999546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.999685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.999779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:57.999939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.000025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.000177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.000255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.000416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.000503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.000645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.external = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.000731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.000870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.000948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.001107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.001196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.001345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.001423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.001581 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.001659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.password = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.001804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.token = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.001883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.002032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.002108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.002260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.002343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.002493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.002572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.002716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.002792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.002956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.003033 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.003193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.003284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.003440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.003530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.003672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.003760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.003917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.003996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.004160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.004241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.004402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.004482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.004633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.004713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.external = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.004869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.004949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.005099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.005177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.005357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.005436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.005606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.005686 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.password = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.005835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] auth.token = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.005913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.006077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.006155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.006314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.006401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.006564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.006649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.006798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.006877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.007047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.007128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.007324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.007409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.007545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.007622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.007777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.007859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.008020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.008108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.008264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.008344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.008492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.008580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.008733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.008811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.008961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.009039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.009195 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.009274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.009429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.009507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.009681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.009760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.009914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.009994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.010154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.010233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.010385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.010464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.010629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.010710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.010873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.010953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.011104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.011183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.011815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012300 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012377 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012377 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.012963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.013109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.013185 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.013287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.013369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.013520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.013611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.013762 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.013846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.014006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.014085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.014257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.014349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.014492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.014572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.014740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.014818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.014975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.015064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.015207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.015287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.015458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.015549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.015690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.015781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.015917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.015996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.016154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.016234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.016389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.016479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.016620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.016697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.016864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.016997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.017077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.017146 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.017321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.017401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.017549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.017627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.017782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.017873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.018021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.018099 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.018262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.018342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.018488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.018575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.018718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.018798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.018943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.019025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.019261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.019435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.019565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.019690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.019813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.019935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.020073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.020197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.020319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.020442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.020577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.020700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.020837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.020960 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.021082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.021204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.021325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.021449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.021580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.021723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.021845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.021968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.022089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.022212 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.022353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.022479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.022565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.022730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.022851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.022973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.023108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.023231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.023340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.023502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.023584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.023750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.023884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.024018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.024144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.024226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.024351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.024469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.024558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.024721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.024843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.024966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.025090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.025221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.025408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.025497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.025603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.025684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.025793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.025882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.025991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026368 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.026914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.027074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.027139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.027255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.027388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.027462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.027462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.027652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.027768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.027919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.028044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.028125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.028253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.028345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.028507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.028633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.028713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.028986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.029109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.029257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.029457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.029580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.029713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.029791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.029853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.030104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.030227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.030361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.030586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.030709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.030843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.031015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.031139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.031274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.031466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.031590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.031789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.031931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.032016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.032248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.032371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.032453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.032659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.032807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.032919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.033050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.033204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.033421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.033499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.033679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.033891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.033980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.034142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.034339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.034424 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.034603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.034734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.034817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.035076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.035160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.035359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.035512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.035597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.035755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.035838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.035998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.036082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.036309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.036446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.036521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.036642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.036721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.036793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.036977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.037063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.037234 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.037318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.037436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.037511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.037702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.037789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.037939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.038022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.038177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.038262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.038413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.038497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.038660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.038744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.038875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.038954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.039120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.039200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.039390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.039471 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.039634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.039725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.039879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.039959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.040124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.040327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.040408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.040565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.040647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.040814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.040895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.041053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.041132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.041297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.041377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.041543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.041621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.041771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.041851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.042023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.042103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.042270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.042454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.042535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.042829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.042908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.042977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.043232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.043546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.043631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.043804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.043899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.044054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.044133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.044289 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.044369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.044523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.044612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.044752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.044832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.044986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.045066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.045221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.045302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.045466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.045547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.045698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.045777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.045933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.046014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.046185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.046266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.046418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.046497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.046658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.046740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.046900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.046979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.047135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.047213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.047397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.047479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.047633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.047716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.047863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.047941 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.048107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.048190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.048342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.048420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.048567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.048645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.048806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.048887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.049057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.049136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.049302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.049382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.049547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.049637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.049798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.049879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.050034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.050114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.050271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.050349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.050503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.050583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.050733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.050811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.050963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.051043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.051194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.051275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.051459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.051538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.051688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.051766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.051927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.052008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.052158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.052248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.052400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.052480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.052630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.052718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.052867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.052948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.053093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.053173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.053327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.053416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.053557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.053637 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.053710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 02 03:19:58.053836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.054081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.054317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.054559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.054794 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.055043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.055283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.055547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.055781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.056022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.056255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 02 03:19:58.056425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80354) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 02 03:19:58.085839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.086134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.086298 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.086876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.087202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.087348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.095342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.096120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.124312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.124539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.127236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.127500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.266941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.270462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.315576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.315806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.317815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.318051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.380373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.380578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.380805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.381023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.381222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.381461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.385711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.385936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.386135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.386473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.386692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.386930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.406921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.407135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.407425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.407635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.407762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.407832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.407914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.408039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.408127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.408286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.408410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.408492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.408700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.408912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.420112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.420239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.433193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.433686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.433995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434673 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.434979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.435121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80354) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 02 03:19:58.453041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.453226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 02 03:19:58.453588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.453833 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 02 03:19:58.454058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.454321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.454577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.454809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.455331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.455666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 02 03:19:58.456121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.456409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 02 03:19:58.456655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.456907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.457152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.457394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.466204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.466486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.466756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.467037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.468712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.468987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.469256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.469521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.475287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.477694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.478584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.478862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.479142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.479450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.479736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.479999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.480793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.481092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.481216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.481412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.481601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.481697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.481867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.481993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.482119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.482119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.482366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.482654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.482922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.483170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.483462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.483723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.508802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.509971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.510421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.510699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.511683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.512143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.516149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.516432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.516712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.518024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.518216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.518626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.524130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.524413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.525830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.526106 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.529045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.529330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.529613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.529892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.530162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.530436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.530660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.530787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.530918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.531002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.531203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.531504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.531771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.532036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.532280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.532535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.549793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.550072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.551446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.551717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.554690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.556366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.559217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.559535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.560333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.560618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.564683 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.565675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.568315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.569243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.571991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.572288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.572570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.572862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.572985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.573126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.573227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.573382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.573472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.573687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.574099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.574309 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.593230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.593356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.593486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.593571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.593745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.593870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.594006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.594086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.601787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.602003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.603495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.603663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.603749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.603952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.606984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.607257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.607553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.607759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.607882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.608081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.608221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.608360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.608585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.608860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.621222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.621469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.621766 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.622038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.622322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.622623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.622903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.623183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.623494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.623775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.623897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.624023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.624224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.624349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.642052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.644096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.645263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.647355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.648572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.650287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.650690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.651902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.652243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.652557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.652813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.654591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.654876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.655157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.659916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.661506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.662519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.664158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.664753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.665042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.665424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.665698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.667397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.667756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.668572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.668870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.674996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.675280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.675550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.678086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.678372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.678661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.683821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.684097 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.684377 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.684649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.684914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.685291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.685569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.685839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.686117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.686378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.686633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.686888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.687153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.687428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80354) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 02 03:19:58.687515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.687707 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.687988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.688244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.688514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.688791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.689069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.689338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.689721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.689994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.690270 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.690534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.690798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.691072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80354) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 02 03:19:58.725357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5642a73669c0 pid: 80353 (default app) Jul 02 03:19:58.729201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7fba2-b9d0-428b-863c-1a25723c04bf None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:19:58.729371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5642a73669c0 pid: 80354 (default app) Jul 02 03:19:58.729677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7fba2-b9d0-428b-863c-1a25723c04bf None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:19:58.730189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7fba2-b9d0-428b-863c-1a25723c04bf None None] PATH_INFO: `/v3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:19:58.736948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Sat Jul 2 03:19:58 2022] GET /identity/v3/ => generated 255 bytes in 11 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:06.331070 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9113ade-22de-4bca-be99-c8749470c70e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:06.332033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9113ade-22de-4bca-be99-c8749470c70e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:06.332154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9113ade-22de-4bca-be99-c8749470c70e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:06.339460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:06 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:06.348350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:06.348768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:06.348921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:06.419830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: INFO dbcounter [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] Registered counter for database keystone Jul 02 03:20:06.444477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG oslo_db.sqlalchemy.engines [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80353) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 02 03:20:06.457653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writer thread running {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 02 03:20:06.640044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG passlib.handlers.bcrypt [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80353) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 02 03:20:06.640467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG passlib.handlers.bcrypt [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80353) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 02 03:20:06.695702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:06.697245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:06.828086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 02 03:20:07.039560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9113ade-22de-4bca-be99-c8749470c70e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:07.117722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: INFO dbcounter [None req-d9113ade-22de-4bca-be99-c8749470c70e None None] Registered counter for database keystone Jul 02 03:20:07.138130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG oslo_db.sqlalchemy.engines [None req-d9113ade-22de-4bca-be99-c8749470c70e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80354) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 02 03:20:07.149521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writer thread running {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 02 03:20:07.267738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9113ade-22de-4bca-be99-c8749470c70e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:07.273330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42740f76-c98f-411e-91f7-a74ad48fded1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:07.273993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42740f76-c98f-411e-91f7-a74ad48fded1 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:07.274282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42740f76-c98f-411e-91f7-a74ad48fded1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:07.274563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42740f76-c98f-411e-91f7-a74ad48fded1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:07.283268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42740f76-c98f-411e-91f7-a74ad48fded1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:07.335590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: WARNING keystone.common.rbac_enforcer.enforcer [None req-42740f76-c98f-411e-91f7-a74ad48fded1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 02 03:20:07.336148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42740f76-c98f-411e-91f7-a74ad48fded1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:07.346324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Sat Jul 2 03:20:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:20:07.353424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:07.369440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e4fe6-e8df-471c-aa8a-3552f61df8b3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:07.376474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc8968-5d8c-442f-ba7f-0411a6498286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:07.377588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc8968-5d8c-442f-ba7f-0411a6498286 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:07.377872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc8968-5d8c-442f-ba7f-0411a6498286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:07.378146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc8968-5d8c-442f-ba7f-0411a6498286 admin admin] PATH_INFO: `/v3/projects/08e65693c4ea4b43b13797fe93e3a44a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:07.382440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc8968-5d8c-442f-ba7f-0411a6498286 admin admin] RBAC: Authorizing `identity:get_project(project_id=08e65693c4ea4b43b13797fe93e3a44a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:07.433534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6bfc8968-5d8c-442f-ba7f-0411a6498286 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 02 03:20:07.433867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc8968-5d8c-442f-ba7f-0411a6498286 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:07.438549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Sat Jul 2 03:20:07 2022] GET /identity/v3/projects/08e65693c4ea4b43b13797fe93e3a44a => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:09.469435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239553f-d58d-4fab-9754-b00d7f1e2cd7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:09.470230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239553f-d58d-4fab-9754-b00d7f1e2cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:09.470230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239553f-d58d-4fab-9754-b00d7f1e2cd7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:09.477029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:09.482732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969907d3-9aa7-468f-b291-0737a484ca55 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:09.483144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969907d3-9aa7-468f-b291-0737a484ca55 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:09.483253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969907d3-9aa7-468f-b291-0737a484ca55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:09.503622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-969907d3-9aa7-468f-b291-0737a484ca55 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:09.505125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-969907d3-9aa7-468f-b291-0737a484ca55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:09.513606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 02 03:20:09.536790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239553f-d58d-4fab-9754-b00d7f1e2cd7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:09.559659 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239553f-d58d-4fab-9754-b00d7f1e2cd7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:09.565524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e14b3d-08b7-4aa1-b01d-5cb5fd6634fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:09.566173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e14b3d-08b7-4aa1-b01d-5cb5fd6634fd admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:09.566460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e14b3d-08b7-4aa1-b01d-5cb5fd6634fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:09.566744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e14b3d-08b7-4aa1-b01d-5cb5fd6634fd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:09.574997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e14b3d-08b7-4aa1-b01d-5cb5fd6634fd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:09.576849 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e14b3d-08b7-4aa1-b01d-5cb5fd6634fd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:09.585965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Sat Jul 2 03:20:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:20:09.594529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969907d3-9aa7-468f-b291-0737a484ca55 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:09.607801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969907d3-9aa7-468f-b291-0737a484ca55 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:09.613832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c59676-d400-469b-8219-75ad99d9ad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:09.614700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c59676-d400-469b-8219-75ad99d9ad48 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:09.614980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c59676-d400-469b-8219-75ad99d9ad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:09.615252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c59676-d400-469b-8219-75ad99d9ad48 admin admin] PATH_INFO: `/v3/users/c350860378554603affd063cfacfc453` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:09.616880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c59676-d400-469b-8219-75ad99d9ad48 admin admin] RBAC: Authorizing `identity:get_user(user_id=c350860378554603affd063cfacfc453)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:09.618726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c59676-d400-469b-8219-75ad99d9ad48 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:09.620334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Sat Jul 2 03:20:09 2022] GET /identity/v3/users/c350860378554603affd063cfacfc453 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:11.707226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fff0-05c9-4cb9-b752-00d57f578b14 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:11.707791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fff0-05c9-4cb9-b752-00d57f578b14 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:11.707791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fff0-05c9-4cb9-b752-00d57f578b14 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:11.709205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:11.714551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8f02f-4b63-4c47-a292-5bfec79edd3f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:11.714994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8f02f-4b63-4c47-a292-5bfec79edd3f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:11.715321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8f02f-4b63-4c47-a292-5bfec79edd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:11.726475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eb8f8-2cf2-4316-976a-08d751d91cf5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:11.726752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eb8f8-2cf2-4316-976a-08d751d91cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:11.727062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eb8f8-2cf2-4316-976a-08d751d91cf5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:11.728432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:11.732383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ccf8f02f-4b63-4c47-a292-5bfec79edd3f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:11.733388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ccf8f02f-4b63-4c47-a292-5bfec79edd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:11.733514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b30239-5d64-4f62-a783-fa042bf3c46b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:11.733790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b30239-5d64-4f62-a783-fa042bf3c46b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:11.734075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b30239-5d64-4f62-a783-fa042bf3c46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:11.775472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG passlib.handlers.bcrypt [None req-b4b30239-5d64-4f62-a783-fa042bf3c46b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80354) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 02 03:20:11.775900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG passlib.handlers.bcrypt [None req-b4b30239-5d64-4f62-a783-fa042bf3c46b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80354) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 02 03:20:11.815860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:11.823198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b4b30239-5d64-4f62-a783-fa042bf3c46b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:11.824861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b4b30239-5d64-4f62-a783-fa042bf3c46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:11.838858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8f02f-4b63-4c47-a292-5bfec79edd3f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:11.863113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:11.926265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:11.945821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8f02f-4b63-4c47-a292-5bfec79edd3f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:11.948839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b30239-5d64-4f62-a783-fa042bf3c46b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:11.986670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3debad-c932-4932-ac5a-f404fb8867e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:11.987874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3debad-c932-4932-ac5a-f404fb8867e4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:11.988283 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3debad-c932-4932-ac5a-f404fb8867e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:11.988700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3debad-c932-4932-ac5a-f404fb8867e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:12.006442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3debad-c932-4932-ac5a-f404fb8867e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-fe3debad-c932-4932-ac5a-f404fb8867e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:12.019558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:12.020448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:12.021268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:20:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:12.026674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3debad-c932-4932-ac5a-f404fb8867e4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:12.063781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b30239-5d64-4f62-a783-fa042bf3c46b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:12.066024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:12.103033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b50b8-b746-45aa-8c5d-cbe2b358930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:12.103970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b50b8-b746-45aa-8c5d-cbe2b358930c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:12.104263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b50b8-b746-45aa-8c5d-cbe2b358930c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:12.104548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b50b8-b746-45aa-8c5d-cbe2b358930c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:12.110394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:12.132052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3debad-c932-4932-ac5a-f404fb8867e4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:12.171753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03512e21-8352-462a-a263-0385212242b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:12.172422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03512e21-8352-462a-a263-0385212242b6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:12.172699 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03512e21-8352-462a-a263-0385212242b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:12.172972 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03512e21-8352-462a-a263-0385212242b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:12.173721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03512e21-8352-462a-a263-0385212242b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:12.174032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03512e21-8352-462a-a263-0385212242b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:12.175481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03512e21-8352-462a-a263-0385212242b6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:12.176142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b50b8-b746-45aa-8c5d-cbe2b358930c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:12.177676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b50b8-b746-45aa-8c5d-cbe2b358930c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:12.196368 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:12.204706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03512e21-8352-462a-a263-0385212242b6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:12.291340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 02 03:20:12.297100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b50b8-b746-45aa-8c5d-cbe2b358930c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:12.299805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:12.309514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03512e21-8352-462a-a263-0385212242b6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:12.343031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22140b-0087-4644-952e-5d8daa3fa0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:12.343944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22140b-0087-4644-952e-5d8daa3fa0ff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:12.344227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22140b-0087-4644-952e-5d8daa3fa0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:12.344505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22140b-0087-4644-952e-5d8daa3fa0ff None admin] PATH_INFO: `/v3/users/c350860378554603affd063cfacfc453` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:12.345793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22140b-0087-4644-952e-5d8daa3fa0ff None admin] RBAC: Authorizing `identity:get_user(user_id=c350860378554603affd063cfacfc453)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:12.347314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22140b-0087-4644-952e-5d8daa3fa0ff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:12.348978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:12 2022] GET /identity/v3/users/c350860378554603affd063cfacfc453 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:12.354683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22140b-0087-4644-952e-5d8daa3fa0ff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:12.369053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:12.394014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b50b8-b746-45aa-8c5d-cbe2b358930c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:12.426313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd258a-bb89-49e7-b44d-d7ac697220ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:12.427105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd258a-bb89-49e7-b44d-d7ac697220ff None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:12.427420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd258a-bb89-49e7-b44d-d7ac697220ff None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:12.427695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd258a-bb89-49e7-b44d-d7ac697220ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:12.447328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd258a-bb89-49e7-b44d-d7ac697220ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:12.448826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd258a-bb89-49e7-b44d-d7ac697220ff None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:12.455697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22140b-0087-4644-952e-5d8daa3fa0ff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-60cd258a-bb89-49e7-b44d-d7ac697220ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:20:12.468773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:12.469590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:12.470575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:20:12.470575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:20:12.470575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:20:12.470575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:12.470575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:12.475785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd258a-bb89-49e7-b44d-d7ac697220ff None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:12.491321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff84ae6-d7bd-4aad-8044-8850c6ded0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:12.492179 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff84ae6-d7bd-4aad-8044-8850c6ded0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:12.492455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff84ae6-d7bd-4aad-8044-8850c6ded0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:12.492730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff84ae6-d7bd-4aad-8044-8850c6ded0d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:12.493916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff84ae6-d7bd-4aad-8044-8850c6ded0d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:12.495486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff84ae6-d7bd-4aad-8044-8850c6ded0d6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:12.497250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:12.502984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff84ae6-d7bd-4aad-8044-8850c6ded0d6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:12.532052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:12.577167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd258a-bb89-49e7-b44d-d7ac697220ff None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:12.583765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:12.603356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff84ae6-d7bd-4aad-8044-8850c6ded0d6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:12.610436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bed79c-48f2-4232-b0ee-fca0144a50bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:12.611107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bed79c-48f2-4232-b0ee-fca0144a50bf None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:12.611414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bed79c-48f2-4232-b0ee-fca0144a50bf None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:12.611688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bed79c-48f2-4232-b0ee-fca0144a50bf None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:12.612412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bed79c-48f2-4232-b0ee-fca0144a50bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:12.612720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bed79c-48f2-4232-b0ee-fca0144a50bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:12.614157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bed79c-48f2-4232-b0ee-fca0144a50bf None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:12.631022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:12.636765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8db60-6799-434f-a642-627cffe32f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:12.637457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8db60-6799-434f-a642-627cffe32f3b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:12.637738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8db60-6799-434f-a642-627cffe32f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:12.638032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8db60-6799-434f-a642-627cffe32f3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:12.638863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8db60-6799-434f-a642-627cffe32f3b None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.domain.id=default, user.id=c350860378554603affd063cfacfc453) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:12.639175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8db60-6799-434f-a642-627cffe32f3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:12.640706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8db60-6799-434f-a642-627cffe32f3b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:12.648511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 11/21] 10.222.0.42 () {56 vars in 1322 bytes} [Sat Jul 2 03:20:12 2022] GET /identity/v3/role_assignments?user.id=c350860378554603affd063cfacfc453&scope.domain.id=default&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:14.927826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37509f-b410-4b2d-b584-c9c21347bab2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:14.928341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37509f-b410-4b2d-b584-c9c21347bab2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:14.928414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37509f-b410-4b2d-b584-c9c21347bab2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:14.929907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:14.945966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f438b-a3b1-4bd7-aa76-ecef90f96db6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:14.946870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f438b-a3b1-4bd7-aa76-ecef90f96db6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:14.947371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f438b-a3b1-4bd7-aa76-ecef90f96db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:14.972596 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1f8f438b-a3b1-4bd7-aa76-ecef90f96db6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:14.974265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1f8f438b-a3b1-4bd7-aa76-ecef90f96db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:15.066698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 12/23] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:15.101324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37509f-b410-4b2d-b584-c9c21347bab2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:15.264004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37509f-b410-4b2d-b584-c9c21347bab2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:15.285290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:15.304826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5423508-e1f5-42f5-b259-fcfa7b2b61c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:15.305625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5423508-e1f5-42f5-b259-fcfa7b2b61c9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.305907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5423508-e1f5-42f5-b259-fcfa7b2b61c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.306182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5423508-e1f5-42f5-b259-fcfa7b2b61c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.328161 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5423508-e1f5-42f5-b259-fcfa7b2b61c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f5423508-e1f5-42f5-b259-fcfa7b2b61c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:15.337291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:15.338117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:15.338880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 12/24] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:20:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:15.344711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f438b-a3b1-4bd7-aa76-ecef90f96db6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:15.469156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:15.502886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f438b-a3b1-4bd7-aa76-ecef90f96db6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:15.554840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d21de-5b9b-4344-917b-1dad60ccb354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:15.555465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d21de-5b9b-4344-917b-1dad60ccb354 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.555752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d21de-5b9b-4344-917b-1dad60ccb354 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.556025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d21de-5b9b-4344-917b-1dad60ccb354 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.557189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d21de-5b9b-4344-917b-1dad60ccb354 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:15.557501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d21de-5b9b-4344-917b-1dad60ccb354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:15.558639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf708c2-730a-42be-a51a-bb1bf8b2a0e9 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.558931 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf708c2-730a-42be-a51a-bb1bf8b2a0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.559044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d21de-5b9b-4344-917b-1dad60ccb354 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:15.559208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf708c2-730a-42be-a51a-bb1bf8b2a0e9 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.560743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 13/25] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:20:15.565797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2913648-24da-4da8-8753-d60ed956ade0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.566243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2913648-24da-4da8-8753-d60ed956ade0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.567118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2913648-24da-4da8-8753-d60ed956ade0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.576685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 13/26] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:15.592380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d21de-5b9b-4344-917b-1dad60ccb354 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:15.614051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a2913648-24da-4da8-8753-d60ed956ade0 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:15.616512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a2913648-24da-4da8-8753-d60ed956ade0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:15.722522 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d21de-5b9b-4344-917b-1dad60ccb354 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:15.764259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8189a2-49d4-40d2-97c9-660bd96542dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:15.765638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8189a2-49d4-40d2-97c9-660bd96542dc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.766109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8189a2-49d4-40d2-97c9-660bd96542dc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.766569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8189a2-49d4-40d2-97c9-660bd96542dc None admin] PATH_INFO: `/v3/users/c350860378554603affd063cfacfc453` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.766681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 14/27] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:15.768154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8189a2-49d4-40d2-97c9-660bd96542dc None admin] RBAC: Authorizing `identity:get_user(user_id=c350860378554603affd063cfacfc453)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:15.768259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f202b-af9a-4658-969c-6662f0e58b1c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.768457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f202b-af9a-4658-969c-6662f0e58b1c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.768958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f202b-af9a-4658-969c-6662f0e58b1c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.769954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8189a2-49d4-40d2-97c9-660bd96542dc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:15.770980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 15/28] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:15.771724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 14/29] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:15 2022] GET /identity/v3/users/c350860378554603affd063cfacfc453 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:15.776204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9a2cb-06bf-4e55-b1e9-2ea7a0e88a8a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.776784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9a2cb-06bf-4e55-b1e9-2ea7a0e88a8a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.776928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc83ceb-50fc-4787-a1f4-8c0b9d0d02bb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.777345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9a2cb-06bf-4e55-b1e9-2ea7a0e88a8a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.777525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc83ceb-50fc-4787-a1f4-8c0b9d0d02bb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.777963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc83ceb-50fc-4787-a1f4-8c0b9d0d02bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.778871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 16/30] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:15.781005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9a2cb-06bf-4e55-b1e9-2ea7a0e88a8a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:15.784813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:15.799814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3bc83ceb-50fc-4787-a1f4-8c0b9d0d02bb None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:15.801325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3bc83ceb-50fc-4787-a1f4-8c0b9d0d02bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:15.888424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 15/31] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:15.894680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd8e47-ed6e-45fb-b32f-0187cf917413 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.894680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd8e47-ed6e-45fb-b32f-0187cf917413 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.894680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd8e47-ed6e-45fb-b32f-0187cf917413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.906648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9a2cb-06bf-4e55-b1e9-2ea7a0e88a8a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:15.915273 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:15.926032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a3fd8e47-ed6e-45fb-b32f-0187cf917413 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:15.927481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a3fd8e47-ed6e-45fb-b32f-0187cf917413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:15.951499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5782d5-6f88-40bc-b5c7-0bf24c7aa50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:15.952679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5782d5-6f88-40bc-b5c7-0bf24c7aa50a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:15.953057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5782d5-6f88-40bc-b5c7-0bf24c7aa50a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:15.953427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5782d5-6f88-40bc-b5c7-0bf24c7aa50a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:15.955493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5782d5-6f88-40bc-b5c7-0bf24c7aa50a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:15.957465 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5782d5-6f88-40bc-b5c7-0bf24c7aa50a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:15.964455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 17/32] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:15.964455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5782d5-6f88-40bc-b5c7-0bf24c7aa50a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:16.008094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 16/33] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:16.009681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd8e47-ed6e-45fb-b32f-0187cf917413 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:16.078380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:16.086612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5782d5-6f88-40bc-b5c7-0bf24c7aa50a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:16.127167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd8e47-ed6e-45fb-b32f-0187cf917413 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:16.131969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde2b1a-a96c-49bf-be77-c2bd2b064582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:16.132658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde2b1a-a96c-49bf-be77-c2bd2b064582 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.132944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde2b1a-a96c-49bf-be77-c2bd2b064582 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.133218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cde2b1a-a96c-49bf-be77-c2bd2b064582 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.133983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde2b1a-a96c-49bf-be77-c2bd2b064582 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:16.135421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cde2b1a-a96c-49bf-be77-c2bd2b064582 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:16.168281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ef912-89b5-4e93-bb83-8572eb6a613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:16.169277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ef912-89b5-4e93-bb83-8572eb6a613c None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.169745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ef912-89b5-4e93-bb83-8572eb6a613c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.170216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ef912-89b5-4e93-bb83-8572eb6a613c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.171213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ef912-89b5-4e93-bb83-8572eb6a613c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:16.172929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ef912-89b5-4e93-bb83-8572eb6a613c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:16.268071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 18/34] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Jul 2 03:20:15 2022] POST /identity/v3/roles => generated 225 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:16.276636 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde2b1a-a96c-49bf-be77-c2bd2b064582 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:16.307550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 17/35] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Jul 2 03:20:16 2022] POST /identity/v3/roles => generated 219 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:16.309102 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ef912-89b5-4e93-bb83-8572eb6a613c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:16.324108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:16.473520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cde2b1a-a96c-49bf-be77-c2bd2b064582 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:16.506855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ef912-89b5-4e93-bb83-8572eb6a613c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:16.514648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f665b4-20c0-4229-a974-3e4df5be2fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:16.516374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f665b4-20c0-4229-a974-3e4df5be2fdc None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.516839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f665b4-20c0-4229-a974-3e4df5be2fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.517274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f665b4-20c0-4229-a974-3e4df5be2fdc None admin] PATH_INFO: `/v3/domains/default/users/c350860378554603affd063cfacfc453/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.521622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f665b4-20c0-4229-a974-3e4df5be2fdc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c350860378554603affd063cfacfc453, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:16.524145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f665b4-20c0-4229-a974-3e4df5be2fdc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:16.534717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:16.548922 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 19/36] 10.222.0.42 () {58 vars in 1361 bytes} [Sat Jul 2 03:20:16 2022] PUT /identity/v3/domains/default/users/c350860378554603affd063cfacfc453/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:20:16.550214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837eae6c-9e0b-4dd3-ae0c-4d8373a0b4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:16.550472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac6797-bd5d-4171-a1af-fcaf1ff7dbef None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.550916 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac6797-bd5d-4171-a1af-fcaf1ff7dbef None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.551486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac6797-bd5d-4171-a1af-fcaf1ff7dbef None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.551677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837eae6c-9e0b-4dd3-ae0c-4d8373a0b4af None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.551974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837eae6c-9e0b-4dd3-ae0c-4d8373a0b4af None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.552255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837eae6c-9e0b-4dd3-ae0c-4d8373a0b4af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.553741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 20/37] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:16.557361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168cec-f4aa-4d05-ac3d-da39f2f8022e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.557860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168cec-f4aa-4d05-ac3d-da39f2f8022e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.558314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168cec-f4aa-4d05-ac3d-da39f2f8022e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.559883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 21/38] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:16.561154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924b1eb-929a-4632-8d4a-37fc7fe8a937 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.561444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924b1eb-929a-4632-8d4a-37fc7fe8a937 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.561735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924b1eb-929a-4632-8d4a-37fc7fe8a937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.569564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837eae6c-9e0b-4dd3-ae0c-4d8373a0b4af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:16.571407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837eae6c-9e0b-4dd3-ae0c-4d8373a0b4af None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:16.588842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3924b1eb-929a-4632-8d4a-37fc7fe8a937 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:16.591343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3924b1eb-929a-4632-8d4a-37fc7fe8a937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-837eae6c-9e0b-4dd3-ae0c-4d8373a0b4af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:20:16.594260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:16.595075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:16.596120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:20:16.596120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:20:16.596120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:20:16.596120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:16.596120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 18/39] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:16.597322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51cd56-7b25-466f-8807-709ca11da834 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.597693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51cd56-7b25-466f-8807-709ca11da834 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.597977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51cd56-7b25-466f-8807-709ca11da834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.617679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6d51cd56-7b25-466f-8807-709ca11da834 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:16.619286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6d51cd56-7b25-466f-8807-709ca11da834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:16.669482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 22/40] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:16.670417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924b1eb-929a-4632-8d4a-37fc7fe8a937 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:16.704241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 19/41] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:16.706086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:16.712336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51cd56-7b25-466f-8807-709ca11da834 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:16.791225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924b1eb-929a-4632-8d4a-37fc7fe8a937 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:16.823164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51cd56-7b25-466f-8807-709ca11da834 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:16.826290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2881e1-8d1f-4434-92d7-3c810a406f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:16.827042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2881e1-8d1f-4434-92d7-3c810a406f82 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.827426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2881e1-8d1f-4434-92d7-3c810a406f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.827774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2881e1-8d1f-4434-92d7-3c810a406f82 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.828581 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2881e1-8d1f-4434-92d7-3c810a406f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:16.828962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2881e1-8d1f-4434-92d7-3c810a406f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:16.830561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2881e1-8d1f-4434-92d7-3c810a406f82 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:16.848412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 23/42] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:16.848412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2881e1-8d1f-4434-92d7-3c810a406f82 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:16.951689 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:16.954913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:16.970354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d206c-87fe-4435-adcf-05a1145537fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:16.971042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d206c-87fe-4435-adcf-05a1145537fa None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:16.971356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d206c-87fe-4435-adcf-05a1145537fa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:16.971643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d206c-87fe-4435-adcf-05a1145537fa None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:16.972394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d206c-87fe-4435-adcf-05a1145537fa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:16.973795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d206c-87fe-4435-adcf-05a1145537fa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:16.973984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2881e1-8d1f-4434-92d7-3c810a406f82 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:16.988159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 20/43] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Jul 2 03:20:16 2022] POST /identity/v3/roles => generated 223 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:17.000345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d206c-87fe-4435-adcf-05a1145537fa None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:17.014250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e09daa-6045-4a53-b2ee-4f5b4544457d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:17.014667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e09daa-6045-4a53-b2ee-4f5b4544457d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:17.014953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e09daa-6045-4a53-b2ee-4f5b4544457d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:17.015228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e09daa-6045-4a53-b2ee-4f5b4544457d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:17.016735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e09daa-6045-4a53-b2ee-4f5b4544457d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:17.018247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e09daa-6045-4a53-b2ee-4f5b4544457d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:17.020041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 24/44] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:17.026227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e09daa-6045-4a53-b2ee-4f5b4544457d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:17.138381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d206c-87fe-4435-adcf-05a1145537fa None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:17.141259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e09daa-6045-4a53-b2ee-4f5b4544457d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:17.178749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fd99e-d2b5-4780-bd3d-76d8cbdd677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:17.179499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fd99e-d2b5-4780-bd3d-76d8cbdd677e None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:17.179789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fd99e-d2b5-4780-bd3d-76d8cbdd677e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:17.180072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fd99e-d2b5-4780-bd3d-76d8cbdd677e None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:17.181180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fd99e-d2b5-4780-bd3d-76d8cbdd677e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:17.181282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13974437-c7b5-4760-8282-488e8863bf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:17.182138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13974437-c7b5-4760-8282-488e8863bf34 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:17.182427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13974437-c7b5-4760-8282-488e8863bf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:17.182741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fd99e-d2b5-4780-bd3d-76d8cbdd677e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:17.182868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13974437-c7b5-4760-8282-488e8863bf34 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:17.183915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13974437-c7b5-4760-8282-488e8863bf34 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:17.186519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13974437-c7b5-4760-8282-488e8863bf34 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:17.217556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 25/45] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:20:17 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:17.221848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 21/46] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:20:16 2022] POST /identity/v3/projects => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:18.669999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdd3e6-640a-4400-8326-44c3ec344c4b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:18.670540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdd3e6-640a-4400-8326-44c3ec344c4b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:18.670540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdd3e6-640a-4400-8326-44c3ec344c4b None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:18.671957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 26/47] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:18.677391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086192b9-cdce-49e6-9c41-88f7df35c51f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:18.677808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086192b9-cdce-49e6-9c41-88f7df35c51f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:18.677954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086192b9-cdce-49e6-9c41-88f7df35c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:18.696742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-086192b9-cdce-49e6-9c41-88f7df35c51f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:18.698081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-086192b9-cdce-49e6-9c41-88f7df35c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:18.712188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:18.773661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 22/48] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:18.796680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdd3e6-640a-4400-8326-44c3ec344c4b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:18.802268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:18.902267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdd3e6-640a-4400-8326-44c3ec344c4b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:18.936978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2c902-1464-4bea-9a7f-9ecbd135b17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:18.937690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2c902-1464-4bea-9a7f-9ecbd135b17e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:18.937974 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2c902-1464-4bea-9a7f-9ecbd135b17e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:18.938250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2c902-1464-4bea-9a7f-9ecbd135b17e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:18.949089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2c902-1464-4bea-9a7f-9ecbd135b17e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0df2c902-1464-4bea-9a7f-9ecbd135b17e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:18.956606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:18.957497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:18.958497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 27/49] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:20:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:18.963789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086192b9-cdce-49e6-9c41-88f7df35c51f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:19.065197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086192b9-cdce-49e6-9c41-88f7df35c51f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:19.099213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8542e7-2b74-4495-ad9a-130fe1fa4369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:19.099909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8542e7-2b74-4495-ad9a-130fe1fa4369 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:19.100188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8542e7-2b74-4495-ad9a-130fe1fa4369 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:19.100460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8542e7-2b74-4495-ad9a-130fe1fa4369 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:19.101213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8542e7-2b74-4495-ad9a-130fe1fa4369 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:19.101517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8542e7-2b74-4495-ad9a-130fe1fa4369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:19.102919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8542e7-2b74-4495-ad9a-130fe1fa4369 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:19.114049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 23/50] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:19.119572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2c902-1464-4bea-9a7f-9ecbd135b17e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:19.186105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:19.230193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2c902-1464-4bea-9a7f-9ecbd135b17e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:19.249697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14f6b2-fc85-49c9-81df-ba85ee006222 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:19.250052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14f6b2-fc85-49c9-81df-ba85ee006222 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:19.250416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14f6b2-fc85-49c9-81df-ba85ee006222 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:19.252599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 24/51] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:19.257893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54349a5d-6767-419c-8685-865f207f794f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:19.258314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54349a5d-6767-419c-8685-865f207f794f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:19.258719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54349a5d-6767-419c-8685-865f207f794f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:19.270220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b6fc2-28b2-4ef3-bf3a-4294d793bf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:19.271096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b6fc2-28b2-4ef3-bf3a-4294d793bf82 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:19.271407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b6fc2-28b2-4ef3-bf3a-4294d793bf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:19.271684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b6fc2-28b2-4ef3-bf3a-4294d793bf82 None admin] PATH_INFO: `/v3/users/c350860378554603affd063cfacfc453` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:19.272951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b6fc2-28b2-4ef3-bf3a-4294d793bf82 None admin] RBAC: Authorizing `identity:get_user(user_id=c350860378554603affd063cfacfc453)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:19.274423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b6fc2-28b2-4ef3-bf3a-4294d793bf82 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:19.276089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 28/52] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:19 2022] GET /identity/v3/users/c350860378554603affd063cfacfc453 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:19.277777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-54349a5d-6767-419c-8685-865f207f794f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:19.278976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-54349a5d-6767-419c-8685-865f207f794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:19.281847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b6fc2-28b2-4ef3-bf3a-4294d793bf82 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:19.282075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:19.347202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 25/53] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:19.370452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54349a5d-6767-419c-8685-865f207f794f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:19.378510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b6fc2-28b2-4ef3-bf3a-4294d793bf82 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:19.407780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:19.413320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7597d4-fdf2-4caa-9e7e-f152937983a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:19.414117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7597d4-fdf2-4caa-9e7e-f152937983a1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:19.414401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7597d4-fdf2-4caa-9e7e-f152937983a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:19.414683 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7597d4-fdf2-4caa-9e7e-f152937983a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:19.415908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7597d4-fdf2-4caa-9e7e-f152937983a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:19.417462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7597d4-fdf2-4caa-9e7e-f152937983a1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:19.419272 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 29/54] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:19.425027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7597d4-fdf2-4caa-9e7e-f152937983a1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:19.468594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54349a5d-6767-419c-8685-865f207f794f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:19.501927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da53c1-6e4e-4086-b078-9e8034c84f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:19.502654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da53c1-6e4e-4086-b078-9e8034c84f17 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:19.502938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da53c1-6e4e-4086-b078-9e8034c84f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:19.503211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da53c1-6e4e-4086-b078-9e8034c84f17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:19.504483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da53c1-6e4e-4086-b078-9e8034c84f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:19.505994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da53c1-6e4e-4086-b078-9e8034c84f17 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:19.507946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 26/55] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:19.514230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da53c1-6e4e-4086-b078-9e8034c84f17 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:19.525951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:19.625127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da53c1-6e4e-4086-b078-9e8034c84f17 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:19.633580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7597d4-fdf2-4caa-9e7e-f152937983a1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:19.657826 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b3f66-6545-411b-8061-9f0d14762e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:19.658481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b3f66-6545-411b-8061-9f0d14762e98 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:19.658782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b3f66-6545-411b-8061-9f0d14762e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:19.659048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b3f66-6545-411b-8061-9f0d14762e98 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:19.659792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b3f66-6545-411b-8061-9f0d14762e98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:19.661265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b3f66-6545-411b-8061-9f0d14762e98 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:19.667540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9f06a-678d-47ad-9269-3471638cde41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:19.668249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9f06a-678d-47ad-9269-3471638cde41 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:19.668532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9f06a-678d-47ad-9269-3471638cde41 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:19.668812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9f06a-678d-47ad-9269-3471638cde41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:19.669645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9f06a-678d-47ad-9269-3471638cde41 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.domain.id=default, user.id=c350860378554603affd063cfacfc453) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:19.669953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9f06a-678d-47ad-9269-3471638cde41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:19.671421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9f06a-678d-47ad-9269-3471638cde41 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:19.679710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 30/56] 10.222.0.42 () {56 vars in 1322 bytes} [Sat Jul 2 03:20:19 2022] GET /identity/v3/role_assignments?user.id=c350860378554603affd063cfacfc453&scope.domain.id=default&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 530 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:19.687388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 27/57] 10.222.0.42 () {60 vars in 1140 bytes} [Sat Jul 2 03:20:19 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:20:21.773231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375a538-b27b-4863-a337-23114f5a1fbc None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:21.773773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375a538-b27b-4863-a337-23114f5a1fbc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:21.773773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375a538-b27b-4863-a337-23114f5a1fbc None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:21.775194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 31/58] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:21.782048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788e262-d781-437a-99f2-81ff907c1da2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:21.782753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788e262-d781-437a-99f2-81ff907c1da2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:21.782941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788e262-d781-437a-99f2-81ff907c1da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:21.805229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0788e262-d781-437a-99f2-81ff907c1da2 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:21.806401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0788e262-d781-437a-99f2-81ff907c1da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:21.869868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:21.882477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 28/59] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:21.907039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375a538-b27b-4863-a337-23114f5a1fbc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:21.981783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:22.022163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375a538-b27b-4863-a337-23114f5a1fbc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:22.062227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c96c8d-1f29-4735-9c7f-ae32e4218700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:22.062958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c96c8d-1f29-4735-9c7f-ae32e4218700 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:22.063241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c96c8d-1f29-4735-9c7f-ae32e4218700 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:22.063554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c96c8d-1f29-4735-9c7f-ae32e4218700 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:22.064900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c96c8d-1f29-4735-9c7f-ae32e4218700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:22.066395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c96c8d-1f29-4735-9c7f-ae32e4218700 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:22.068073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 32/60] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:22.074001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788e262-d781-437a-99f2-81ff907c1da2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:22.180254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0788e262-d781-437a-99f2-81ff907c1da2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:22.216085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8b54a-cc83-4b1c-b35a-fc74d1db77e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:22.216750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8b54a-cc83-4b1c-b35a-fc74d1db77e1 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:22.217032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8b54a-cc83-4b1c-b35a-fc74d1db77e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:22.217305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8b54a-cc83-4b1c-b35a-fc74d1db77e1 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:22.218017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8b54a-cc83-4b1c-b35a-fc74d1db77e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:22.219493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8b54a-cc83-4b1c-b35a-fc74d1db77e1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:22.246005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 29/61] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:20:22 2022] POST /identity/v3/users => generated 275 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:25.355877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365af4d-349a-4332-b2ea-01750e567271 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.356925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365af4d-349a-4332-b2ea-01750e567271 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.357406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365af4d-349a-4332-b2ea-01750e567271 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.359243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 33/62] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:25.365213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb79e38-9a65-49f7-a3e0-4a20f59f2346 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.365609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb79e38-9a65-49f7-a3e0-4a20f59f2346 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.365982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb79e38-9a65-49f7-a3e0-4a20f59f2346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.392365 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ffb79e38-9a65-49f7-a3e0-4a20f59f2346 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:25.393575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ffb79e38-9a65-49f7-a3e0-4a20f59f2346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:25.451806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:25.487666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 30/63] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:25.520990 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365af4d-349a-4332-b2ea-01750e567271 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:25.537096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7ff1e-fe0a-4b6d-9c49-2321f9022795 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.537602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7ff1e-fe0a-4b6d-9c49-2321f9022795 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.538017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7ff1e-fe0a-4b6d-9c49-2321f9022795 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.539589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 31/64] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:20:25.544923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acef2f3-34d6-4661-b090-31b662d781e2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.545409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acef2f3-34d6-4661-b090-31b662d781e2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.545822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acef2f3-34d6-4661-b090-31b662d781e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.562829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3acef2f3-34d6-4661-b090-31b662d781e2 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:25.564250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3acef2f3-34d6-4661-b090-31b662d781e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:25.682315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 32/65] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:25.687236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365af4d-349a-4332-b2ea-01750e567271 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:25.707097 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acef2f3-34d6-4661-b090-31b662d781e2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:25.755162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:25.765035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2a9bc-aafd-4660-9d56-efb22b8c36bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:25.765845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2a9bc-aafd-4660-9d56-efb22b8c36bd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.766132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2a9bc-aafd-4660-9d56-efb22b8c36bd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.766408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2a9bc-aafd-4660-9d56-efb22b8c36bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.786421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d2a9bc-aafd-4660-9d56-efb22b8c36bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d0d2a9bc-aafd-4660-9d56-efb22b8c36bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:25.797188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:25.798060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:25.798892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 34/66] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:25.799716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9010f1-e4b7-4e18-b6a0-0cd977b878dd None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.800171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9010f1-e4b7-4e18-b6a0-0cd977b878dd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.800595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9010f1-e4b7-4e18-b6a0-0cd977b878dd None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.802279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 35/67] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:25.808223 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9010f1-e4b7-4e18-b6a0-0cd977b878dd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:25.845554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acef2f3-34d6-4661-b090-31b662d781e2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:25.855552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:25.886292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59238d-bfc8-4ef1-93e6-fbe823feac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:25.887261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59238d-bfc8-4ef1-93e6-fbe823feac1d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.887636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59238d-bfc8-4ef1-93e6-fbe823feac1d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.887970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59238d-bfc8-4ef1-93e6-fbe823feac1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.899608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59238d-bfc8-4ef1-93e6-fbe823feac1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0a59238d-bfc8-4ef1-93e6-fbe823feac1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:25.908312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:25.909232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:25.909994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:25.911266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 33/68] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:25.912671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7c6a5-ea12-4b7c-80ac-47b34efcd0eb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.912965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7c6a5-ea12-4b7c-80ac-47b34efcd0eb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.913250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7c6a5-ea12-4b7c-80ac-47b34efcd0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.931025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a1a7c6a5-ea12-4b7c-80ac-47b34efcd0eb None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:25.932806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a1a7c6a5-ea12-4b7c-80ac-47b34efcd0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:25.936246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9010f1-e4b7-4e18-b6a0-0cd977b878dd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:25.971235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e19bab-9657-4c8c-8a9c-95f9eb41255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:25.972003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e19bab-9657-4c8c-8a9c-95f9eb41255d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.972293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e19bab-9657-4c8c-8a9c-95f9eb41255d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.972576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e19bab-9657-4c8c-8a9c-95f9eb41255d None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.973328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e19bab-9657-4c8c-8a9c-95f9eb41255d None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:25.973640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e19bab-9657-4c8c-8a9c-95f9eb41255d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:25.975039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e19bab-9657-4c8c-8a9c-95f9eb41255d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:25.988059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 36/69] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:25.989285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710a7cf-8ada-4c69-ba69-87cbfa2a486d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:25.989575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710a7cf-8ada-4c69-ba69-87cbfa2a486d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:25.989856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710a7cf-8ada-4c69-ba69-87cbfa2a486d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:25.991246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 37/70] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:25.991818 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710a7cf-8ada-4c69-ba69-87cbfa2a486d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:26.023127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 34/71] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:26.023127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7c6a5-ea12-4b7c-80ac-47b34efcd0eb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:26.064174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:26.112271 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710a7cf-8ada-4c69-ba69-87cbfa2a486d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:26.118043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:26.124277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a7c6a5-ea12-4b7c-80ac-47b34efcd0eb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:26.147920 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900332b-be1c-4cae-84fc-5a7d913528c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:26.148603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900332b-be1c-4cae-84fc-5a7d913528c4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.148889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900332b-be1c-4cae-84fc-5a7d913528c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.149170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900332b-be1c-4cae-84fc-5a7d913528c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.149927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900332b-be1c-4cae-84fc-5a7d913528c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:26.150239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900332b-be1c-4cae-84fc-5a7d913528c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:26.151713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900332b-be1c-4cae-84fc-5a7d913528c4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:26.158359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57386f43-1bec-4020-a1a2-d2584c5ef993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:26.159242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57386f43-1bec-4020-a1a2-d2584c5ef993 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.159558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57386f43-1bec-4020-a1a2-d2584c5ef993 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.159754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 38/72] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:26.159836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57386f43-1bec-4020-a1a2-d2584c5ef993 None admin] PATH_INFO: `/v3/users/80cfaef5476c4d42b916f2cf9c155962` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.160942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2581477-be57-43ae-8484-5d21ca1d466c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.161231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2581477-be57-43ae-8484-5d21ca1d466c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.161622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2581477-be57-43ae-8484-5d21ca1d466c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.173302 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57386f43-1bec-4020-a1a2-d2584c5ef993 None admin] RBAC: Authorizing `identity:get_user(user_id=80cfaef5476c4d42b916f2cf9c155962)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:26.174801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57386f43-1bec-4020-a1a2-d2584c5ef993 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:26.176556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 35/73] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:26 2022] GET /identity/v3/users/80cfaef5476c4d42b916f2cf9c155962 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:26.177638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e2581477-be57-43ae-8484-5d21ca1d466c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:26.178857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e2581477-be57-43ae-8484-5d21ca1d466c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:26.179193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3fcf-fc80-4468-9f83-49cd82526c51 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.179526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3fcf-fc80-4468-9f83-49cd82526c51 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.179806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef3fcf-fc80-4468-9f83-49cd82526c51 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.181368 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 36/74] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:26.184118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3fcf-fc80-4468-9f83-49cd82526c51 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:26.247217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 39/75] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:26.259463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2581477-be57-43ae-8484-5d21ca1d466c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:26.294833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef3fcf-fc80-4468-9f83-49cd82526c51 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:26.297801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:26.331981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabae73e-835d-4aa8-9b52-0e3d93e0b370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:26.332776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae73e-835d-4aa8-9b52-0e3d93e0b370 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.333081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae73e-835d-4aa8-9b52-0e3d93e0b370 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.333366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabae73e-835d-4aa8-9b52-0e3d93e0b370 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.334590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabae73e-835d-4aa8-9b52-0e3d93e0b370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:26.336082 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabae73e-835d-4aa8-9b52-0e3d93e0b370 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:26.337935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 37/76] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:26.338485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabae73e-835d-4aa8-9b52-0e3d93e0b370 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:26.357033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2581477-be57-43ae-8484-5d21ca1d466c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:26.364087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:26.392274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f74ed-2ff5-4c9f-88a8-b1f94d6ae1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:26.393155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f74ed-2ff5-4c9f-88a8-b1f94d6ae1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.393442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f74ed-2ff5-4c9f-88a8-b1f94d6ae1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.393708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f74ed-2ff5-4c9f-88a8-b1f94d6ae1a6 None admin] PATH_INFO: `/v3/users/c350860378554603affd063cfacfc453` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.394925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f74ed-2ff5-4c9f-88a8-b1f94d6ae1a6 None admin] RBAC: Authorizing `identity:get_user(user_id=c350860378554603affd063cfacfc453)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:26.396431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f74ed-2ff5-4c9f-88a8-b1f94d6ae1a6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:26.398107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 40/77] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:26 2022] GET /identity/v3/users/c350860378554603affd063cfacfc453 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:26.401044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c61a5c-e363-4b89-bdeb-9b87b71f0562 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.401044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c61a5c-e363-4b89-bdeb-9b87b71f0562 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.401044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c61a5c-e363-4b89-bdeb-9b87b71f0562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.416151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-21c61a5c-e363-4b89-bdeb-9b87b71f0562 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:26.417954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-21c61a5c-e363-4b89-bdeb-9b87b71f0562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:26.444070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabae73e-835d-4aa8-9b52-0e3d93e0b370 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:26.478671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee189a4-9d02-4593-952f-a22fb6343033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:26.479584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee189a4-9d02-4593-952f-a22fb6343033 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.479880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee189a4-9d02-4593-952f-a22fb6343033 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.480323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee189a4-9d02-4593-952f-a22fb6343033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.485234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 41/78] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:26.486068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c61a5c-e363-4b89-bdeb-9b87b71f0562 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:26.536311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:26.550798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee189a4-9d02-4593-952f-a22fb6343033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:26.552415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee189a4-9d02-4593-952f-a22fb6343033 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:26.575528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:26.590757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c61a5c-e363-4b89-bdeb-9b87b71f0562 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:26.624163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4cb07-dbb0-4f92-8f67-fd6167c23226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:26.624977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4cb07-dbb0-4f92-8f67-fd6167c23226 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.625261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4cb07-dbb0-4f92-8f67-fd6167c23226 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.625540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4cb07-dbb0-4f92-8f67-fd6167c23226 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.636811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4cb07-dbb0-4f92-8f67-fd6167c23226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4ce4cb07-dbb0-4f92-8f67-fd6167c23226 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:26.644795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:26.645802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 42/79] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:20:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:26.652062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4cb07-dbb0-4f92-8f67-fd6167c23226 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:26.662105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 38/80] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:26.662961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee189a4-9d02-4593-952f-a22fb6343033 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:26.750797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4cb07-dbb0-4f92-8f67-fd6167c23226 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:26.754032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:26.763082 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee189a4-9d02-4593-952f-a22fb6343033 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:26.786345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:26.786938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfbf7d-e67a-4605-a8fb-ff6a5313029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:26.787652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfbf7d-e67a-4605-a8fb-ff6a5313029d None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.788043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfbf7d-e67a-4605-a8fb-ff6a5313029d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.788324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfbf7d-e67a-4605-a8fb-ff6a5313029d None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.789076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfbf7d-e67a-4605-a8fb-ff6a5313029d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:26.790552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfbf7d-e67a-4605-a8fb-ff6a5313029d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:26.799555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9aab-db66-44fc-ba21-f508387b0013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:26.800210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9aab-db66-44fc-ba21-f508387b0013 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.800495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9aab-db66-44fc-ba21-f508387b0013 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.800776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9aab-db66-44fc-ba21-f508387b0013 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:26.818097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 43/81] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:20:26 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:26.818787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfbf7d-e67a-4605-a8fb-ff6a5313029d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:26.888537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9aab-db66-44fc-ba21-f508387b0013 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:26.891149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9aab-db66-44fc-ba21-f508387b0013 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:26.954121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfbf7d-e67a-4605-a8fb-ff6a5313029d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:26.996532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea658-0aea-41bc-a120-8b1c330c1388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:26.997372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea658-0aea-41bc-a120-8b1c330c1388 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:26.997655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea658-0aea-41bc-a120-8b1c330c1388 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:26.998230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea658-0aea-41bc-a120-8b1c330c1388 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:27.009088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 39/82] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:27.009525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ea658-0aea-41bc-a120-8b1c330c1388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:27.009878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9aab-db66-44fc-ba21-f508387b0013 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-eb5ea658-0aea-41bc-a120-8b1c330c1388 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:27.019036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:27.020111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:27.020876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 44/83] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:27.021493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea658-0aea-41bc-a120-8b1c330c1388 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.037746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:27.048551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:27.141391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9aab-db66-44fc-ba21-f508387b0013 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:27.156045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea658-0aea-41bc-a120-8b1c330c1388 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:27.181425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7ee19-9d34-47c2-b360-58cdec28a32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:27.182432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7ee19-9d34-47c2-b360-58cdec28a32f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:27.182905 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7ee19-9d34-47c2-b360-58cdec28a32f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:27.183621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7ee19-9d34-47c2-b360-58cdec28a32f None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:27.184467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7ee19-9d34-47c2-b360-58cdec28a32f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:27.184851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7ee19-9d34-47c2-b360-58cdec28a32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:27.186334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7ee19-9d34-47c2-b360-58cdec28a32f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:27.199380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 40/84] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:27.199380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7ee19-9d34-47c2-b360-58cdec28a32f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.205126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ba61a-265b-4428-8577-6fc58bde18bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:27.206195 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ba61a-265b-4428-8577-6fc58bde18bb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:27.207040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ba61a-265b-4428-8577-6fc58bde18bb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:27.207488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ba61a-265b-4428-8577-6fc58bde18bb None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:27.209357 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ba61a-265b-4428-8577-6fc58bde18bb None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:27.211831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ba61a-265b-4428-8577-6fc58bde18bb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:27.214109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 45/85] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:27.215479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ba61a-265b-4428-8577-6fc58bde18bb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.311327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:27.320239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:27.329109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7ee19-9d34-47c2-b360-58cdec28a32f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:27.337544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ba61a-265b-4428-8577-6fc58bde18bb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:27.365573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236d2ca-c878-467f-8f27-06300a93b7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:27.366611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236d2ca-c878-467f-8f27-06300a93b7fc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:27.367050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236d2ca-c878-467f-8f27-06300a93b7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:27.367509 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236d2ca-c878-467f-8f27-06300a93b7fc None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:27.368965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236d2ca-c878-467f-8f27-06300a93b7fc None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:27.371413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236d2ca-c878-467f-8f27-06300a93b7fc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:27.374076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 41/86] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:27.375094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236d2ca-c878-467f-8f27-06300a93b7fc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.381883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f04cc-e243-4a00-966c-c9b7b7dcb05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:27.383090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f04cc-e243-4a00-966c-c9b7b7dcb05f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:27.383551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f04cc-e243-4a00-966c-c9b7b7dcb05f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:27.383968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f04cc-e243-4a00-966c-c9b7b7dcb05f None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:27.384915 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f04cc-e243-4a00-966c-c9b7b7dcb05f None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:27.385364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f04cc-e243-4a00-966c-c9b7b7dcb05f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:27.387595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f04cc-e243-4a00-966c-c9b7b7dcb05f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:27.399535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 46/87] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:27.400690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f04cc-e243-4a00-966c-c9b7b7dcb05f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.490643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236d2ca-c878-467f-8f27-06300a93b7fc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:27.524885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6f44f-55ba-49ac-bb30-8c1b9e51e8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:27.525921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6f44f-55ba-49ac-bb30-8c1b9e51e8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:27.526332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6f44f-55ba-49ac-bb30-8c1b9e51e8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:27.526750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6f44f-55ba-49ac-bb30-8c1b9e51e8c4 None admin] PATH_INFO: `/v3/users/80cfaef5476c4d42b916f2cf9c155962` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:27.528118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6f44f-55ba-49ac-bb30-8c1b9e51e8c4 None admin] RBAC: Authorizing `identity:get_user(user_id=80cfaef5476c4d42b916f2cf9c155962)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:27.529681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6f44f-55ba-49ac-bb30-8c1b9e51e8c4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:27.531544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 42/88] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/users/80cfaef5476c4d42b916f2cf9c155962 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:27.536263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6f44f-55ba-49ac-bb30-8c1b9e51e8c4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.536380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f04cc-e243-4a00-966c-c9b7b7dcb05f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:27.572079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:27.584793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aca770-a117-4af1-be32-bd3ea0f89d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:27.585965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aca770-a117-4af1-be32-bd3ea0f89d0a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:27.586601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aca770-a117-4af1-be32-bd3ea0f89d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:27.587078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aca770-a117-4af1-be32-bd3ea0f89d0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:27.588327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aca770-a117-4af1-be32-bd3ea0f89d0a None admin] RBAC: Adding query filter params (role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=94c3ce30be2e489e9210838c8132fd92, user.id=80cfaef5476c4d42b916f2cf9c155962) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:27.588838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aca770-a117-4af1-be32-bd3ea0f89d0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:27.590914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aca770-a117-4af1-be32-bd3ea0f89d0a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:27.599097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 47/89] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/role_assignments?user.id=80cfaef5476c4d42b916f2cf9c155962&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:27.605017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aca770-a117-4af1-be32-bd3ea0f89d0a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.642821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:27.687025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6f44f-55ba-49ac-bb30-8c1b9e51e8c4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:27.767113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aca770-a117-4af1-be32-bd3ea0f89d0a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:27.787541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16c1ee-9a91-438c-a783-14344ecc15df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:27.788229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16c1ee-9a91-438c-a783-14344ecc15df None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:27.788518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16c1ee-9a91-438c-a783-14344ecc15df None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:27.788799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16c1ee-9a91-438c-a783-14344ecc15df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:27.789715 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16c1ee-9a91-438c-a783-14344ecc15df None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=94c3ce30be2e489e9210838c8132fd92, user.id=c350860378554603affd063cfacfc453) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:27.790034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16c1ee-9a91-438c-a783-14344ecc15df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:27.791724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16c1ee-9a91-438c-a783-14344ecc15df None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:27.809045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 43/90] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/role_assignments?user.id=c350860378554603affd063cfacfc453&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:27.809045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16c1ee-9a91-438c-a783-14344ecc15df None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.819488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84e5b7-77e3-4434-b168-eff1791fc1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:27.820643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84e5b7-77e3-4434-b168-eff1791fc1ce None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:27.821055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84e5b7-77e3-4434-b168-eff1791fc1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:27.821490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84e5b7-77e3-4434-b168-eff1791fc1ce None admin] PATH_INFO: `/v3/users/80cfaef5476c4d42b916f2cf9c155962` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:27.823504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84e5b7-77e3-4434-b168-eff1791fc1ce None admin] RBAC: Authorizing `identity:get_user(user_id=80cfaef5476c4d42b916f2cf9c155962)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:27.828193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84e5b7-77e3-4434-b168-eff1791fc1ce None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:27.830044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 48/91] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/users/80cfaef5476c4d42b916f2cf9c155962 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:27.835947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84e5b7-77e3-4434-b168-eff1791fc1ce None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:27.931974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:27.940859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:27.986453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84e5b7-77e3-4434-b168-eff1791fc1ce None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:27.991281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16c1ee-9a91-438c-a783-14344ecc15df None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:28.044326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0d8e4-5f33-4793-bc99-03e949a42032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:28.045003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0d8e4-5f33-4793-bc99-03e949a42032 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:28.045286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0d8e4-5f33-4793-bc99-03e949a42032 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:28.045649 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0d8e4-5f33-4793-bc99-03e949a42032 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:28.076213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c54e30-aff6-42cb-ba07-e5899459063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:28.077241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c54e30-aff6-42cb-ba07-e5899459063c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:28.077725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c54e30-aff6-42cb-ba07-e5899459063c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:28.078216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c54e30-aff6-42cb-ba07-e5899459063c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:28.176315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0d8e4-5f33-4793-bc99-03e949a42032 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:28.177840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0d8e4-5f33-4793-bc99-03e949a42032 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:28.220618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c54e30-aff6-42cb-ba07-e5899459063c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:28.222593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c54e30-aff6-42cb-ba07-e5899459063c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:28.274804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:28.309452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 49/92] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:28.315260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0d8e4-5f33-4793-bc99-03e949a42032 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:28.389177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:28.418539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 44/93] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:28.424994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c54e30-aff6-42cb-ba07-e5899459063c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:28.437755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0d8e4-5f33-4793-bc99-03e949a42032 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:28.481996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea694-ad82-48f7-b3a5-79ce2b9fc945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:28.483747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea694-ad82-48f7-b3a5-79ce2b9fc945 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:28.484243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea694-ad82-48f7-b3a5-79ce2b9fc945 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:28.485170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea694-ad82-48f7-b3a5-79ce2b9fc945 None admin] PATH_INFO: `/v3/projects/bade203bac76419a9e736d5b2e23806b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:28.487012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dea694-ad82-48f7-b3a5-79ce2b9fc945 None admin] RBAC: Authorizing `identity:get_project(project_id=bade203bac76419a9e736d5b2e23806b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:28.489260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dea694-ad82-48f7-b3a5-79ce2b9fc945 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:28.491951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 50/94] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:28 2022] GET /identity/v3/projects/bade203bac76419a9e736d5b2e23806b => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:28.500334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea694-ad82-48f7-b3a5-79ce2b9fc945 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:28.595706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c54e30-aff6-42cb-ba07-e5899459063c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:28.599492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:28.661241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dea694-ad82-48f7-b3a5-79ce2b9fc945 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:28.672084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ef2bf-da49-4fdd-88c1-13103cd663ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:28.673233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ef2bf-da49-4fdd-88c1-13103cd663ed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:28.673750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ef2bf-da49-4fdd-88c1-13103cd663ed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:28.674215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ef2bf-da49-4fdd-88c1-13103cd663ed None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:28.675617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ef2bf-da49-4fdd-88c1-13103cd663ed None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:28.677261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ef2bf-da49-4fdd-88c1-13103cd663ed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:28.679173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 45/95] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:28 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:28.688317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ef2bf-da49-4fdd-88c1-13103cd663ed None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:28.738664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f72212-a67a-4ece-b35d-f8efe43b9e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:28.739353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72212-a67a-4ece-b35d-f8efe43b9e36 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:28.739639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72212-a67a-4ece-b35d-f8efe43b9e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:28.739917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f72212-a67a-4ece-b35d-f8efe43b9e36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:28.740762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72212-a67a-4ece-b35d-f8efe43b9e36 None admin] RBAC: Adding query filter params (role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=bade203bac76419a9e736d5b2e23806b, user.id=80cfaef5476c4d42b916f2cf9c155962) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:28.741079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72212-a67a-4ece-b35d-f8efe43b9e36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:28.747602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f72212-a67a-4ece-b35d-f8efe43b9e36 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:28.760633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 51/96] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:28 2022] GET /identity/v3/role_assignments?user.id=80cfaef5476c4d42b916f2cf9c155962&scope.project.id=bade203bac76419a9e736d5b2e23806b&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:28.796708 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:28.867544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ef2bf-da49-4fdd-88c1-13103cd663ed None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:28.915525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08466d-5ea4-4a5d-ab03-e88742e23ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:28.916119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08466d-5ea4-4a5d-ab03-e88742e23ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:28.916404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08466d-5ea4-4a5d-ab03-e88742e23ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:28.916683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08466d-5ea4-4a5d-ab03-e88742e23ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:28.917571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08466d-5ea4-4a5d-ab03-e88742e23ba9 None admin] RBAC: Adding query filter params (role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=94c3ce30be2e489e9210838c8132fd92, user.id=80cfaef5476c4d42b916f2cf9c155962) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:28.917887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08466d-5ea4-4a5d-ab03-e88742e23ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:28.919398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08466d-5ea4-4a5d-ab03-e88742e23ba9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:28.932782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 46/97] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:28 2022] GET /identity/v3/role_assignments?user.id=80cfaef5476c4d42b916f2cf9c155962&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=62ac4326d6d742c5b6684746953362b1 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:29.692603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460763b-6e94-4e88-9ed7-44bdc24ed693 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:29.693122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460763b-6e94-4e88-9ed7-44bdc24ed693 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:29.693194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460763b-6e94-4e88-9ed7-44bdc24ed693 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:29.694540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 52/98] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:29.700250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f71ab8-a3ec-436d-8a14-776f4375e484 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:29.701017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f71ab8-a3ec-436d-8a14-776f4375e484 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:29.701462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f71ab8-a3ec-436d-8a14-776f4375e484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:29.721393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-08f71ab8-a3ec-436d-8a14-776f4375e484 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:29.722581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08f71ab8-a3ec-436d-8a14-776f4375e484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:29.725801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5b7d2-7234-4a2a-a2e9-e7a0395bfe1c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:29.726309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5b7d2-7234-4a2a-a2e9-e7a0395bfe1c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:29.726806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5b7d2-7234-4a2a-a2e9-e7a0395bfe1c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:29.728431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 53/99] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:20:29.746421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567be30-03b8-46c3-828c-ed009753f199 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:29.747071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567be30-03b8-46c3-828c-ed009753f199 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:29.747588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567be30-03b8-46c3-828c-ed009753f199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:29.784375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a567be30-03b8-46c3-828c-ed009753f199 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:29.785593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a567be30-03b8-46c3-828c-ed009753f199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:29.810676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 47/100] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:29.857296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f71ab8-a3ec-436d-8a14-776f4375e484 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:29.914153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:29.926927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 54/101] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:29.962521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567be30-03b8-46c3-828c-ed009753f199 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:29.992168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f71ab8-a3ec-436d-8a14-776f4375e484 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:30.023713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:30.061569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ab07-58ea-417b-b758-6005e8e61bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:30.062381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ab07-58ea-417b-b758-6005e8e61bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:30.062663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ab07-58ea-417b-b758-6005e8e61bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:30.062937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7ab07-58ea-417b-b758-6005e8e61bb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:30.065471 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7ab07-58ea-417b-b758-6005e8e61bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:30.066959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7ab07-58ea-417b-b758-6005e8e61bb5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:30.077697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 48/102] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:30.078597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ab07-58ea-417b-b758-6005e8e61bb5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:30.133105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567be30-03b8-46c3-828c-ed009753f199 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:30.172449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6fcd8-2991-44dd-8928-b47c8bd0c6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:30.173573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6fcd8-2991-44dd-8928-b47c8bd0c6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:30.174064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6fcd8-2991-44dd-8928-b47c8bd0c6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:30.174475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6fcd8-2991-44dd-8928-b47c8bd0c6c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:30.186363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6fcd8-2991-44dd-8928-b47c8bd0c6c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-06c6fcd8-2991-44dd-8928-b47c8bd0c6c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:30.194739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:30.195800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:30.196711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:30.199448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 55/103] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:30.208663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6fcd8-2991-44dd-8928-b47c8bd0c6c5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:30.329453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7ab07-58ea-417b-b758-6005e8e61bb5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:30.335189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6fcd8-2991-44dd-8928-b47c8bd0c6c5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:30.338343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:30.359597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:30.375419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cab4c-624f-4f5b-8159-a6798d67916a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:30.376315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cab4c-624f-4f5b-8159-a6798d67916a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:30.376985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cab4c-624f-4f5b-8159-a6798d67916a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:30.377398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cab4c-624f-4f5b-8159-a6798d67916a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:30.378541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cab4c-624f-4f5b-8159-a6798d67916a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:30.379012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cab4c-624f-4f5b-8159-a6798d67916a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:30.380582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cab4c-624f-4f5b-8159-a6798d67916a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:30.390613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 56/104] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:30.396902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cab4c-624f-4f5b-8159-a6798d67916a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:30.418050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25418508-2f86-47fa-9593-7cc900b6988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:30.418837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418508-2f86-47fa-9593-7cc900b6988a None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:30.419120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418508-2f86-47fa-9593-7cc900b6988a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:30.419422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25418508-2f86-47fa-9593-7cc900b6988a None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:30.420143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25418508-2f86-47fa-9593-7cc900b6988a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:30.421604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25418508-2f86-47fa-9593-7cc900b6988a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:30.487431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 49/105] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:20:30 2022] POST /identity/v3/projects => generated 302 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:30.488952 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef4a82-b1a2-4d2e-90d2-25dcfd3a347e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:30.489416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef4a82-b1a2-4d2e-90d2-25dcfd3a347e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:30.490081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef4a82-b1a2-4d2e-90d2-25dcfd3a347e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:30.490979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 50/106] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:30.512897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24614c-67a9-47cd-9e74-7af28ab90986 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:30.513188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24614c-67a9-47cd-9e74-7af28ab90986 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:30.513470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24614c-67a9-47cd-9e74-7af28ab90986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:30.556348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8b24614c-67a9-47cd-9e74-7af28ab90986 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:30.557951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8b24614c-67a9-47cd-9e74-7af28ab90986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:30.561884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cab4c-624f-4f5b-8159-a6798d67916a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:30.603554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1365a-316c-410a-8465-9ca8726cccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:30.604839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1365a-316c-410a-8465-9ca8726cccee None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:30.605441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1365a-316c-410a-8465-9ca8726cccee None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:30.605874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1365a-316c-410a-8465-9ca8726cccee None admin] PATH_INFO: `/v3/users/94585aa232a74c0584bdb9b5b044ebd7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:30.625232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1365a-316c-410a-8465-9ca8726cccee None admin] RBAC: Authorizing `identity:get_user(user_id=94585aa232a74c0584bdb9b5b044ebd7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:30.626955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1365a-316c-410a-8465-9ca8726cccee None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:30.629596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 57/107] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:30 2022] GET /identity/v3/users/94585aa232a74c0584bdb9b5b044ebd7 => generated 289 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:30.636743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1365a-316c-410a-8465-9ca8726cccee None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:30.665893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:30.755756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1365a-316c-410a-8465-9ca8726cccee None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:30.789301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 51/108] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:30.801005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee8adb-b19d-42ed-a051-598f2570c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:30.801679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8adb-b19d-42ed-a051-598f2570c88d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:30.801974 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8adb-b19d-42ed-a051-598f2570c88d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:30.802264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8adb-b19d-42ed-a051-598f2570c88d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:30.828181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24614c-67a9-47cd-9e74-7af28ab90986 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:30.879554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee8adb-b19d-42ed-a051-598f2570c88d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:30.881979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee8adb-b19d-42ed-a051-598f2570c88d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:30.948735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:30.956977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24614c-67a9-47cd-9e74-7af28ab90986 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:30.973741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:30.995032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfb09b-b431-4d83-a1bb-b66c923a396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:30.996400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfb09b-b431-4d83-a1bb-b66c923a396e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:30.996677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfb09b-b431-4d83-a1bb-b66c923a396e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:30.997095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfb09b-b431-4d83-a1bb-b66c923a396e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.008966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfb09b-b431-4d83-a1bb-b66c923a396e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:31.015682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 58/109] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c2cfb09b-b431-4d83-a1bb-b66c923a396e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:31.017210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:31.018241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:31.019174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 52/110] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:31.019867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282978e5-5352-4e32-87a7-aa9eac940837 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.019867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282978e5-5352-4e32-87a7-aa9eac940837 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.020214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282978e5-5352-4e32-87a7-aa9eac940837 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.023014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfb09b-b431-4d83-a1bb-b66c923a396e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:31.023148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 59/111] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:31.025102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282978e5-5352-4e32-87a7-aa9eac940837 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:31.163086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfb09b-b431-4d83-a1bb-b66c923a396e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:31.184261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282978e5-5352-4e32-87a7-aa9eac940837 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:31.205882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c124b7-df7a-464f-990c-01972478673f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:31.208220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c124b7-df7a-464f-990c-01972478673f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.210222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c124b7-df7a-464f-990c-01972478673f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.210709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c124b7-df7a-464f-990c-01972478673f None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.213086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c124b7-df7a-464f-990c-01972478673f None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:31.216540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c124b7-df7a-464f-990c-01972478673f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:31.219946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 53/112] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:31 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:31.225518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c37db9-8b2b-445f-92b8-55b493c09eaa None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.226046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c37db9-8b2b-445f-92b8-55b493c09eaa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.226521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c37db9-8b2b-445f-92b8-55b493c09eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.237059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:31.255489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d5c37db9-8b2b-445f-92b8-55b493c09eaa None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:31.255489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d5c37db9-8b2b-445f-92b8-55b493c09eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:31.269043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db1032-8f86-4216-9388-e1db9df87fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:31.269729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db1032-8f86-4216-9388-e1db9df87fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.270058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db1032-8f86-4216-9388-e1db9df87fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.270339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db1032-8f86-4216-9388-e1db9df87fa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.271090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db1032-8f86-4216-9388-e1db9df87fa8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:31.271421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db1032-8f86-4216-9388-e1db9df87fa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:31.272833 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db1032-8f86-4216-9388-e1db9df87fa8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:31.281888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 60/113] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:31.283314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db1032-8f86-4216-9388-e1db9df87fa8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:31.296698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:31.395264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 54/114] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:31.400277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c37db9-8b2b-445f-92b8-55b493c09eaa None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:31.453400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db1032-8f86-4216-9388-e1db9df87fa8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:31.491951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4693a31-476a-49c6-996f-5932a19d2e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:31.492989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4693a31-476a-49c6-996f-5932a19d2e76 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.493275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4693a31-476a-49c6-996f-5932a19d2e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.493551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4693a31-476a-49c6-996f-5932a19d2e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.494447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4693a31-476a-49c6-996f-5932a19d2e76 None admin] RBAC: Adding query filter params (role.id=3ff68e8f3bb340bf985c55b19b58c222, scope.project.id=94c3ce30be2e489e9210838c8132fd92, user.id=94585aa232a74c0584bdb9b5b044ebd7) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:31.494761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4693a31-476a-49c6-996f-5932a19d2e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:31.496402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4693a31-476a-49c6-996f-5932a19d2e76 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:31.504973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 61/115] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:31 2022] GET /identity/v3/role_assignments?user.id=94585aa232a74c0584bdb9b5b044ebd7&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=3ff68e8f3bb340bf985c55b19b58c222 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:31.505938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4693a31-476a-49c6-996f-5932a19d2e76 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:31.613798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c37db9-8b2b-445f-92b8-55b493c09eaa None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:31.652804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4693a31-476a-49c6-996f-5932a19d2e76 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:31.667766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:31.676694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b269fd4-e1c2-49db-b568-4711bb258dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:31.677730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b269fd4-e1c2-49db-b568-4711bb258dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.678025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b269fd4-e1c2-49db-b568-4711bb258dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.678302 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b269fd4-e1c2-49db-b568-4711bb258dc2 None admin] PATH_INFO: `/v3/users/80cfaef5476c4d42b916f2cf9c155962` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.684159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b269fd4-e1c2-49db-b568-4711bb258dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=80cfaef5476c4d42b916f2cf9c155962)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:31.685685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b269fd4-e1c2-49db-b568-4711bb258dc2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:31.687467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 55/116] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:31 2022] GET /identity/v3/users/80cfaef5476c4d42b916f2cf9c155962 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:31.688724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea06668-6962-4aa4-9cbd-093a9991eec8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.689014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea06668-6962-4aa4-9cbd-093a9991eec8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.689293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea06668-6962-4aa4-9cbd-093a9991eec8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.690670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 56/117] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:31.696250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea06668-6962-4aa4-9cbd-093a9991eec8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:31.698314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33125256-fb6a-4f7c-9403-e212bc862a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:31.699642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125256-fb6a-4f7c-9403-e212bc862a0a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.700233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125256-fb6a-4f7c-9403-e212bc862a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.700712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125256-fb6a-4f7c-9403-e212bc862a0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.713214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33125256-fb6a-4f7c-9403-e212bc862a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:31.721666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-33125256-fb6a-4f7c-9403-e212bc862a0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:31.722780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:31.724951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:31.725749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:31.728623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 62/118] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:31.730345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ba21-1671-4801-8097-060fe539f088 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.730956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ba21-1671-4801-8097-060fe539f088 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.731537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164ba21-1671-4801-8097-060fe539f088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.749416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6164ba21-1671-4801-8097-060fe539f088 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:31.751033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6164ba21-1671-4801-8097-060fe539f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:31.819420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea06668-6962-4aa4-9cbd-093a9991eec8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:31.826480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 63/119] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:31.832045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164ba21-1671-4801-8097-060fe539f088 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:31.888703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e43577-0693-485e-8221-09792cb852c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:31.895473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e43577-0693-485e-8221-09792cb852c0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:31.895473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e43577-0693-485e-8221-09792cb852c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:31.895473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e43577-0693-485e-8221-09792cb852c0 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:31.895473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e43577-0693-485e-8221-09792cb852c0 None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:31.895473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e43577-0693-485e-8221-09792cb852c0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:31.895473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 57/120] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:31 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:31.895880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e43577-0693-485e-8221-09792cb852c0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:31.923321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:31.996394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e43577-0693-485e-8221-09792cb852c0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:31.998208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:32.023320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164ba21-1671-4801-8097-060fe539f088 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:32.033883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdecc0f-b6e9-4cc5-9e60-0296db6017ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:32.034783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdecc0f-b6e9-4cc5-9e60-0296db6017ee None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.035141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdecc0f-b6e9-4cc5-9e60-0296db6017ee None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.039336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdecc0f-b6e9-4cc5-9e60-0296db6017ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.046380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdecc0f-b6e9-4cc5-9e60-0296db6017ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-abdecc0f-b6e9-4cc5-9e60-0296db6017ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:32.054381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:32.055975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:32.057538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:32.059704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 58/121] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:32.059704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdecc0f-b6e9-4cc5-9e60-0296db6017ee None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:32.100809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75237a4d-7e24-469e-8cdf-4c3c0c7466af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:32.101497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75237a4d-7e24-469e-8cdf-4c3c0c7466af None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.101781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75237a4d-7e24-469e-8cdf-4c3c0c7466af None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.102059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75237a4d-7e24-469e-8cdf-4c3c0c7466af None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.102816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75237a4d-7e24-469e-8cdf-4c3c0c7466af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:32.103121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75237a4d-7e24-469e-8cdf-4c3c0c7466af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:32.104670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75237a4d-7e24-469e-8cdf-4c3c0c7466af None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:32.123387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 64/122] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:32.124261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75237a4d-7e24-469e-8cdf-4c3c0c7466af None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:32.168450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdecc0f-b6e9-4cc5-9e60-0296db6017ee None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:32.210986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa07cfa-9717-467e-90c3-769f1e38fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:32.212125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07cfa-9717-467e-90c3-769f1e38fa94 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.212621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07cfa-9717-467e-90c3-769f1e38fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.212837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa07cfa-9717-467e-90c3-769f1e38fa94 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92/users/80cfaef5476c4d42b916f2cf9c155962/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.215326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa07cfa-9717-467e-90c3-769f1e38fa94 None admin] RBAC: Authorizing `identity:create_grant(project_id=94c3ce30be2e489e9210838c8132fd92, user_id=80cfaef5476c4d42b916f2cf9c155962, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:32.217075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa07cfa-9717-467e-90c3-769f1e38fa94 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:32.237294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 59/123] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:20:32 2022] PUT /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92/users/80cfaef5476c4d42b916f2cf9c155962/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:20:32.238673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa07cfa-9717-467e-90c3-769f1e38fa94 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:32.257746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:32.280077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75237a4d-7e24-469e-8cdf-4c3c0c7466af None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:32.282238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:32.323471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f614e0e-c219-476f-a176-8f25faa37dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:32.324648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f614e0e-c219-476f-a176-8f25faa37dcb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.325121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f614e0e-c219-476f-a176-8f25faa37dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.325663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f614e0e-c219-476f-a176-8f25faa37dcb None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.326638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f614e0e-c219-476f-a176-8f25faa37dcb None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:32.327154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f614e0e-c219-476f-a176-8f25faa37dcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:32.328978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f614e0e-c219-476f-a176-8f25faa37dcb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:32.338702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 65/124] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:32.340475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216aa86-1193-40e6-a52e-35d8d941e01c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.341025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216aa86-1193-40e6-a52e-35d8d941e01c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.341516 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216aa86-1193-40e6-a52e-35d8d941e01c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.343205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 66/125] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:32.348570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216aa86-1193-40e6-a52e-35d8d941e01c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:32.376422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa07cfa-9717-467e-90c3-769f1e38fa94 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:32.421196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b91a1-4196-4276-ac8a-ff3d09caeea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:32.423031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b91a1-4196-4276-ac8a-ff3d09caeea3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.423770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b91a1-4196-4276-ac8a-ff3d09caeea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.424458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b91a1-4196-4276-ac8a-ff3d09caeea3 None admin] PATH_INFO: `/v3/users/c350860378554603affd063cfacfc453` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.426264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b91a1-4196-4276-ac8a-ff3d09caeea3 None admin] RBAC: Authorizing `identity:get_user(user_id=c350860378554603affd063cfacfc453)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:32.428722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b91a1-4196-4276-ac8a-ff3d09caeea3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:32.432087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 60/126] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:32 2022] GET /identity/v3/users/c350860378554603affd063cfacfc453 => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:32.433727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ce10-f102-454c-86b7-00f617571cb7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.434207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ce10-f102-454c-86b7-00f617571cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.434655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5ce10-f102-454c-86b7-00f617571cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.469746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-50d5ce10-f102-454c-86b7-00f617571cb7 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:32.472844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-50d5ce10-f102-454c-86b7-00f617571cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:32.494414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216aa86-1193-40e6-a52e-35d8d941e01c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:32.541535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb9e01-30e4-4f64-ac55-96e7c7f9db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:32.543237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb9e01-30e4-4f64-ac55-96e7c7f9db9e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.543621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb9e01-30e4-4f64-ac55-96e7c7f9db9e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.543903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb9e01-30e4-4f64-ac55-96e7c7f9db9e None admin] PATH_INFO: `/v3/users/80cfaef5476c4d42b916f2cf9c155962` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.545264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb9e01-30e4-4f64-ac55-96e7c7f9db9e None admin] RBAC: Authorizing `identity:get_user(user_id=80cfaef5476c4d42b916f2cf9c155962)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:32.546748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb9e01-30e4-4f64-ac55-96e7c7f9db9e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:32.551362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 67/127] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:32 2022] GET /identity/v3/users/80cfaef5476c4d42b916f2cf9c155962 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:32.552457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb9e01-30e4-4f64-ac55-96e7c7f9db9e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:32.586363 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 61/128] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:32.591321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ce10-f102-454c-86b7-00f617571cb7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:32.599393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:32.616621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:32.705404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5ce10-f102-454c-86b7-00f617571cb7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:32.743419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24058b-1218-4d70-81d4-229ec9c90560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:32.744430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24058b-1218-4d70-81d4-229ec9c90560 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.744880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24058b-1218-4d70-81d4-229ec9c90560 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.745220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24058b-1218-4d70-81d4-229ec9c90560 None admin] PATH_INFO: `/v3/projects/bade203bac76419a9e736d5b2e23806b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.746406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24058b-1218-4d70-81d4-229ec9c90560 None admin] RBAC: Authorizing `identity:get_project(project_id=bade203bac76419a9e736d5b2e23806b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:32.748024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24058b-1218-4d70-81d4-229ec9c90560 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:32.755049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 62/129] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:32 2022] GET /identity/v3/projects/bade203bac76419a9e736d5b2e23806b => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:32.755896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24058b-1218-4d70-81d4-229ec9c90560 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:32.773272 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb9e01-30e4-4f64-ac55-96e7c7f9db9e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:32.844538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:32.854371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b457b-b591-472c-8a56-a59f28387fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:32.856382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b457b-b591-472c-8a56-a59f28387fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.856672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b457b-b591-472c-8a56-a59f28387fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.856948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b457b-b591-472c-8a56-a59f28387fa8 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.860011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b457b-b591-472c-8a56-a59f28387fa8 None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:32.862983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b457b-b591-472c-8a56-a59f28387fa8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:32.866880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 68/130] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:32 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:32.869544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b457b-b591-472c-8a56-a59f28387fa8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:32.894131 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24058b-1218-4d70-81d4-229ec9c90560 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:32.945941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514edba-62b1-40d6-95ef-59c38a39239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:32.946771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514edba-62b1-40d6-95ef-59c38a39239b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:32.947053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514edba-62b1-40d6-95ef-59c38a39239b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:32.947347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3514edba-62b1-40d6-95ef-59c38a39239b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:32.971971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3514edba-62b1-40d6-95ef-59c38a39239b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3514edba-62b1-40d6-95ef-59c38a39239b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:32.989662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:32.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:32.991481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 63/131] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:20:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:32.992126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514edba-62b1-40d6-95ef-59c38a39239b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:33.063655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:33.069417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b457b-b591-472c-8a56-a59f28387fa8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:33.130511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3514edba-62b1-40d6-95ef-59c38a39239b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:33.164029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530cc05-d4f9-47fa-97f8-b1b1461b048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:33.164953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530cc05-d4f9-47fa-97f8-b1b1461b048d None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:33.165250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530cc05-d4f9-47fa-97f8-b1b1461b048d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:33.165536 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530cc05-d4f9-47fa-97f8-b1b1461b048d None admin] PATH_INFO: `/v3/projects/bade203bac76419a9e736d5b2e23806b/users/80cfaef5476c4d42b916f2cf9c155962/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:33.167071 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e505b2d-914d-4566-87f4-4d001d0f8301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:33.168170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e505b2d-914d-4566-87f4-4d001d0f8301 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:33.168600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e505b2d-914d-4566-87f4-4d001d0f8301 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:33.169020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e505b2d-914d-4566-87f4-4d001d0f8301 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92/users/c350860378554603affd063cfacfc453/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:33.171118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e505b2d-914d-4566-87f4-4d001d0f8301 None admin] RBAC: Authorizing `identity:create_grant(project_id=94c3ce30be2e489e9210838c8132fd92, user_id=c350860378554603affd063cfacfc453, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:33.172119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530cc05-d4f9-47fa-97f8-b1b1461b048d None admin] RBAC: Authorizing `identity:create_grant(project_id=bade203bac76419a9e736d5b2e23806b, user_id=80cfaef5476c4d42b916f2cf9c155962, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:33.173606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530cc05-d4f9-47fa-97f8-b1b1461b048d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:33.173893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e505b2d-914d-4566-87f4-4d001d0f8301 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:33.177753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:33.194728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 64/132] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:20:32 2022] PUT /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92/users/c350860378554603affd063cfacfc453/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:20:33.196076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e505b2d-914d-4566-87f4-4d001d0f8301 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:33.202084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 69/133] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:20:32 2022] PUT /identity/v3/projects/bade203bac76419a9e736d5b2e23806b/users/80cfaef5476c4d42b916f2cf9c155962/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:20:33.340413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e505b2d-914d-4566-87f4-4d001d0f8301 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:33.387542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c7809-79a8-43ad-8420-ccc6d787ede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:33.388563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c7809-79a8-43ad-8420-ccc6d787ede4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:33.389025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c7809-79a8-43ad-8420-ccc6d787ede4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:33.389453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c7809-79a8-43ad-8420-ccc6d787ede4 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:33.390420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c7809-79a8-43ad-8420-ccc6d787ede4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:33.390867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c7809-79a8-43ad-8420-ccc6d787ede4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:33.392417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c7809-79a8-43ad-8420-ccc6d787ede4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:33.403398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 65/134] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:20:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:33.409182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530cc05-d4f9-47fa-97f8-b1b1461b048d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:33.414491 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039262e-989e-4b80-acbc-30571a7401fd None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:33.415019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039262e-989e-4b80-acbc-30571a7401fd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:33.415742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039262e-989e-4b80-acbc-30571a7401fd None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:33.418458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 66/135] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:20:33.424379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb88a1-6bd3-4879-a5bb-ea96d068d465 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:33.425620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb88a1-6bd3-4879-a5bb-ea96d068d465 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:33.426052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb88a1-6bd3-4879-a5bb-ea96d068d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:33.456329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-eaeb88a1-6bd3-4879-a5bb-ea96d068d465 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:33.457796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eaeb88a1-6bd3-4879-a5bb-ea96d068d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:33.543229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:33.559281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 67/136] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:33.585235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb88a1-6bd3-4879-a5bb-ea96d068d465 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:33.608483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530cc05-d4f9-47fa-97f8-b1b1461b048d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:33.656165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78602d4f-ccd3-49fb-ab56-748a2430e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:33.657234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78602d4f-ccd3-49fb-ab56-748a2430e999 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:33.657530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78602d4f-ccd3-49fb-ab56-748a2430e999 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:33.657815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78602d4f-ccd3-49fb-ab56-748a2430e999 None admin] PATH_INFO: `/v3/users/80cfaef5476c4d42b916f2cf9c155962` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:33.659331 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78602d4f-ccd3-49fb-ab56-748a2430e999 None admin] RBAC: Authorizing `identity:get_user(user_id=80cfaef5476c4d42b916f2cf9c155962)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:33.661485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78602d4f-ccd3-49fb-ab56-748a2430e999 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:33.663384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 70/137] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:33 2022] GET /identity/v3/users/80cfaef5476c4d42b916f2cf9c155962 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:33.668903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78602d4f-ccd3-49fb-ab56-748a2430e999 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:33.715430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb88a1-6bd3-4879-a5bb-ea96d068d465 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:33.724423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:33.804990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af3da7-24b4-4850-96ad-4b42aef00a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:33.805806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af3da7-24b4-4850-96ad-4b42aef00a3f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:33.806083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af3da7-24b4-4850-96ad-4b42aef00a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:33.806355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af3da7-24b4-4850-96ad-4b42aef00a3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:33.809072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af3da7-24b4-4850-96ad-4b42aef00a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:33.810761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af3da7-24b4-4850-96ad-4b42aef00a3f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:33.817880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 68/138] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:33.828744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af3da7-24b4-4850-96ad-4b42aef00a3f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:33.892262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78602d4f-ccd3-49fb-ab56-748a2430e999 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:33.963088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:33.967863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf84ac-56a9-4a79-bcaa-c609fcc0d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:33.969193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf84ac-56a9-4a79-bcaa-c609fcc0d978 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:33.969693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf84ac-56a9-4a79-bcaa-c609fcc0d978 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:33.970207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf84ac-56a9-4a79-bcaa-c609fcc0d978 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:33.975744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf84ac-56a9-4a79-bcaa-c609fcc0d978 None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:33.977440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf84ac-56a9-4a79-bcaa-c609fcc0d978 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:33.980030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 71/139] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:33 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:33.989768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf84ac-56a9-4a79-bcaa-c609fcc0d978 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:34.032871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af3da7-24b4-4850-96ad-4b42aef00a3f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:34.095864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28abe23-b051-47ee-bc14-2297b85ecf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:34.096546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28abe23-b051-47ee-bc14-2297b85ecf95 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:34.096828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28abe23-b051-47ee-bc14-2297b85ecf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:34.097104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28abe23-b051-47ee-bc14-2297b85ecf95 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:34.097903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28abe23-b051-47ee-bc14-2297b85ecf95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:34.099379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28abe23-b051-47ee-bc14-2297b85ecf95 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:34.133926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 69/140] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:20:33 2022] POST /identity/v3/users => generated 283 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:34.171135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf84ac-56a9-4a79-bcaa-c609fcc0d978 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:34.179564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:34.225729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1f9d8-4040-4981-a2d4-5a6cf7216180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:34.226559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f9d8-4040-4981-a2d4-5a6cf7216180 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:34.226844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f9d8-4040-4981-a2d4-5a6cf7216180 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:34.227119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1f9d8-4040-4981-a2d4-5a6cf7216180 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92/users/80cfaef5476c4d42b916f2cf9c155962/roles/62ac4326d6d742c5b6684746953362b1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:34.229515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1f9d8-4040-4981-a2d4-5a6cf7216180 None admin] RBAC: Authorizing `identity:create_grant(project_id=94c3ce30be2e489e9210838c8132fd92, user_id=80cfaef5476c4d42b916f2cf9c155962, role_id=62ac4326d6d742c5b6684746953362b1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:34.231026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1f9d8-4040-4981-a2d4-5a6cf7216180 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:34.255693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 72/141] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:20:33 2022] PUT /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92/users/80cfaef5476c4d42b916f2cf9c155962/roles/62ac4326d6d742c5b6684746953362b1 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:20:34.495111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea449-316f-427d-b8dc-26add7570e85 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:34.495532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea449-316f-427d-b8dc-26add7570e85 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:34.495723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea449-316f-427d-b8dc-26add7570e85 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:34.497395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 70/142] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:34.502839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355d560-3486-4cca-be77-47fda0487bac None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:34.503135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355d560-3486-4cca-be77-47fda0487bac None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:34.503447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355d560-3486-4cca-be77-47fda0487bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:34.524759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8355d560-3486-4cca-be77-47fda0487bac None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:34.525943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8355d560-3486-4cca-be77-47fda0487bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:34.607878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 73/143] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:34.632111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdea449-316f-427d-b8dc-26add7570e85 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:34.741856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdea449-316f-427d-b8dc-26add7570e85 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:34.757045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:34.780530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cfc1b-9a49-4e7a-b4c2-b5dc5ee86e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:34.781336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cfc1b-9a49-4e7a-b4c2-b5dc5ee86e4f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:34.781618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cfc1b-9a49-4e7a-b4c2-b5dc5ee86e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:34.781914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4cfc1b-9a49-4e7a-b4c2-b5dc5ee86e4f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:34.793284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4cfc1b-9a49-4e7a-b4c2-b5dc5ee86e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-fc4cfc1b-9a49-4e7a-b4c2-b5dc5ee86e4f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:34.801073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:34.801926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:34.802677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 71/144] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:34.808922 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355d560-3486-4cca-be77-47fda0487bac None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:34.914758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:34.930459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355d560-3486-4cca-be77-47fda0487bac None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:34.971130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927a184-7e39-4194-b89b-2ee7064f65e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:34.971849 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927a184-7e39-4194-b89b-2ee7064f65e7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:34.972135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927a184-7e39-4194-b89b-2ee7064f65e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:34.972410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927a184-7e39-4194-b89b-2ee7064f65e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:34.973158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927a184-7e39-4194-b89b-2ee7064f65e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:34.973465 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927a184-7e39-4194-b89b-2ee7064f65e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:34.974861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927a184-7e39-4194-b89b-2ee7064f65e7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:34.983528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 74/145] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:34.989032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cfc1b-9a49-4e7a-b4c2-b5dc5ee86e4f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:35.043040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5bfe3-c5d9-42a7-97fe-e16e024a1a53 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:35.043381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5bfe3-c5d9-42a7-97fe-e16e024a1a53 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:35.043544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5bfe3-c5d9-42a7-97fe-e16e024a1a53 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:35.045253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 75/146] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:20:35.051096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3398c8-f9a1-45b1-b7e0-c026b9979876 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:35.051436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3398c8-f9a1-45b1-b7e0-c026b9979876 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:35.051717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3398c8-f9a1-45b1-b7e0-c026b9979876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:35.078826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ab3398c8-f9a1-45b1-b7e0-c026b9979876 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:35.080050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ab3398c8-f9a1-45b1-b7e0-c026b9979876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:35.124536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4cfc1b-9a49-4e7a-b4c2-b5dc5ee86e4f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:35.195477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d7445-1845-439a-9c95-f0c17b466b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:35.196370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d7445-1845-439a-9c95-f0c17b466b7f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:35.196649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d7445-1845-439a-9c95-f0c17b466b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:35.196922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d7445-1845-439a-9c95-f0c17b466b7f None admin] PATH_INFO: `/v3/users/94585aa232a74c0584bdb9b5b044ebd7` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:35.199336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d7445-1845-439a-9c95-f0c17b466b7f None admin] RBAC: Authorizing `identity:get_user(user_id=94585aa232a74c0584bdb9b5b044ebd7)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:35.200898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d7445-1845-439a-9c95-f0c17b466b7f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:35.202942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 72/147] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:34 2022] GET /identity/v3/users/94585aa232a74c0584bdb9b5b044ebd7 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:35.207162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 76/148] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:35.211528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d7445-1845-439a-9c95-f0c17b466b7f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:35.273505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:35.276214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3398c8-f9a1-45b1-b7e0-c026b9979876 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:35.358424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d7445-1845-439a-9c95-f0c17b466b7f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:35.453210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c5ad1-2a54-4654-a83a-69de7eaa5b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:35.454599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c5ad1-2a54-4654-a83a-69de7eaa5b1d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:35.455074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c5ad1-2a54-4654-a83a-69de7eaa5b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:35.459854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c5ad1-2a54-4654-a83a-69de7eaa5b1d None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:35.464158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c5ad1-2a54-4654-a83a-69de7eaa5b1d None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:35.466798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c5ad1-2a54-4654-a83a-69de7eaa5b1d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:35.469999 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 73/149] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:35 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:35.479168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c5ad1-2a54-4654-a83a-69de7eaa5b1d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:35.515796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:35.542737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3398c8-f9a1-45b1-b7e0-c026b9979876 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:35.582677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a69353-95ec-4b8a-a131-5c2eb20045dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:35.583504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a69353-95ec-4b8a-a131-5c2eb20045dc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:35.583786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a69353-95ec-4b8a-a131-5c2eb20045dc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:35.584062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a69353-95ec-4b8a-a131-5c2eb20045dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:35.597139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a69353-95ec-4b8a-a131-5c2eb20045dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-87a69353-95ec-4b8a-a131-5c2eb20045dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:35.607208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:35.610078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:35.612047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 77/150] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:35.620902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a69353-95ec-4b8a-a131-5c2eb20045dc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:35.748566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a69353-95ec-4b8a-a131-5c2eb20045dc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:35.784055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd1861-8ee5-4e94-acb6-a6be023398de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:35.784729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd1861-8ee5-4e94-acb6-a6be023398de None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:35.785013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd1861-8ee5-4e94-acb6-a6be023398de None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:35.785292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd1861-8ee5-4e94-acb6-a6be023398de None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:35.786053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd1861-8ee5-4e94-acb6-a6be023398de None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:35.786361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd1861-8ee5-4e94-acb6-a6be023398de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:35.787819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd1861-8ee5-4e94-acb6-a6be023398de None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:35.793585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:35.798542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 78/151] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:35.804091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd1861-8ee5-4e94-acb6-a6be023398de None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:35.830747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c5ad1-2a54-4654-a83a-69de7eaa5b1d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:35.855207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:35.908984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd1861-8ee5-4e94-acb6-a6be023398de None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:35.944773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601dcff-a5e6-4f50-82a0-ee400cb85237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:35.945961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601dcff-a5e6-4f50-82a0-ee400cb85237 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:35.945961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601dcff-a5e6-4f50-82a0-ee400cb85237 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:35.946493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601dcff-a5e6-4f50-82a0-ee400cb85237 None admin] PATH_INFO: `/v3/users/80cfaef5476c4d42b916f2cf9c155962` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:35.947416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601dcff-a5e6-4f50-82a0-ee400cb85237 None admin] RBAC: Authorizing `identity:get_user(user_id=80cfaef5476c4d42b916f2cf9c155962)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:35.949348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601dcff-a5e6-4f50-82a0-ee400cb85237 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:35.951266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 79/152] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:35 2022] GET /identity/v3/users/80cfaef5476c4d42b916f2cf9c155962 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:35.953871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32a7e1-2c30-42b6-9377-d14ae352e63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:35.955683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32a7e1-2c30-42b6-9377-d14ae352e63e None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:35.955931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32a7e1-2c30-42b6-9377-d14ae352e63e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:35.956375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32a7e1-2c30-42b6-9377-d14ae352e63e None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92/users/94585aa232a74c0584bdb9b5b044ebd7/roles/3ff68e8f3bb340bf985c55b19b58c222` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:35.956625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601dcff-a5e6-4f50-82a0-ee400cb85237 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:35.959801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32a7e1-2c30-42b6-9377-d14ae352e63e None admin] RBAC: Authorizing `identity:create_grant(project_id=94c3ce30be2e489e9210838c8132fd92, user_id=94585aa232a74c0584bdb9b5b044ebd7, role_id=3ff68e8f3bb340bf985c55b19b58c222)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:35.962472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf32a7e1-2c30-42b6-9377-d14ae352e63e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:36.016767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 74/153] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:20:35 2022] PUT /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92/users/94585aa232a74c0584bdb9b5b044ebd7/roles/3ff68e8f3bb340bf985c55b19b58c222 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:20:36.027356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:36.064288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601dcff-a5e6-4f50-82a0-ee400cb85237 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:36.112144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bc96-ac12-41f7-ae1d-bba7b0afc0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:36.112144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bc96-ac12-41f7-ae1d-bba7b0afc0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:36.112144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bc96-ac12-41f7-ae1d-bba7b0afc0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:36.112144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9bc96-ac12-41f7-ae1d-bba7b0afc0e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:36.206296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9bc96-ac12-41f7-ae1d-bba7b0afc0e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:36.207726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9bc96-ac12-41f7-ae1d-bba7b0afc0e3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:36.350404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:36.415981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 80/154] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 02 03:20:36.422288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32a7e1-2c30-42b6-9377-d14ae352e63e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:36.608504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32a7e1-2c30-42b6-9377-d14ae352e63e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:36.668818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ff0f7-62d5-4082-a28f-146ece446b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:36.670268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ff0f7-62d5-4082-a28f-146ece446b91 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:36.670747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ff0f7-62d5-4082-a28f-146ece446b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:36.671221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ff0f7-62d5-4082-a28f-146ece446b91 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:36.673307 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ff0f7-62d5-4082-a28f-146ece446b91 None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:36.675957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ff0f7-62d5-4082-a28f-146ece446b91 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:36.683423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 75/155] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:36 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:36.685998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bc96-ac12-41f7-ae1d-bba7b0afc0e3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:36.753379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d1027-8b12-4f7f-a72e-56538f4f3e22 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:36.753877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d1027-8b12-4f7f-a72e-56538f4f3e22 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:36.754271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d1027-8b12-4f7f-a72e-56538f4f3e22 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:36.756819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 76/156] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:36.763904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddc47c-b7ac-4365-bd30-344bae853c77 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:36.764380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddc47c-b7ac-4365-bd30-344bae853c77 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:36.764854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddc47c-b7ac-4365-bd30-344bae853c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:36.811667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-29ddc47c-b7ac-4365-bd30-344bae853c77 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:36.813593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29ddc47c-b7ac-4365-bd30-344bae853c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:36.912371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:36.995060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9bc96-ac12-41f7-ae1d-bba7b0afc0e3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:37.009069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 77/157] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:37.053885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c09fe-eebc-497d-ae4c-fa13fe752d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:37.054548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c09fe-eebc-497d-ae4c-fa13fe752d01 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:37.054837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c09fe-eebc-497d-ae4c-fa13fe752d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:37.055228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c09fe-eebc-497d-ae4c-fa13fe752d01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:37.055978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c09fe-eebc-497d-ae4c-fa13fe752d01 None admin] RBAC: Adding query filter params (role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=94c3ce30be2e489e9210838c8132fd92, user.id=80cfaef5476c4d42b916f2cf9c155962) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:37.056295 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c09fe-eebc-497d-ae4c-fa13fe752d01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:37.057952 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddc47c-b7ac-4365-bd30-344bae853c77 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:37.058616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c09fe-eebc-497d-ae4c-fa13fe752d01 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:37.068126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 81/158] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:36 2022] GET /identity/v3/role_assignments?user.id=80cfaef5476c4d42b916f2cf9c155962&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 608 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:37.268328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddc47c-b7ac-4365-bd30-344bae853c77 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:37.332317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a9ac4-eac4-4129-9c62-7a1f357a2d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:37.333509 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a9ac4-eac4-4129-9c62-7a1f357a2d04 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:37.333955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a9ac4-eac4-4129-9c62-7a1f357a2d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:37.334399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a9ac4-eac4-4129-9c62-7a1f357a2d04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:37.357007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a9ac4-eac4-4129-9c62-7a1f357a2d04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a25a9ac4-eac4-4129-9c62-7a1f357a2d04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:37.376215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:37.377923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:37.379407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 78/159] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:37.384575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c09fe-eebc-497d-ae4c-fa13fe752d01 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:37.442806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:37.527380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c09fe-eebc-497d-ae4c-fa13fe752d01 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:37.570110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab901abc-5ac5-4c85-8cf9-e634fe17a393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:37.570792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab901abc-5ac5-4c85-8cf9-e634fe17a393 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:37.571079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab901abc-5ac5-4c85-8cf9-e634fe17a393 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:37.571384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab901abc-5ac5-4c85-8cf9-e634fe17a393 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:37.572136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab901abc-5ac5-4c85-8cf9-e634fe17a393 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:37.572446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab901abc-5ac5-4c85-8cf9-e634fe17a393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:37.573977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab901abc-5ac5-4c85-8cf9-e634fe17a393 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:37.587008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 82/160] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:37.592810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a9ac4-eac4-4129-9c62-7a1f357a2d04 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:37.610025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22a9d0-9d04-4cf2-9e28-db7d2f363e46 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:37.610522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22a9d0-9d04-4cf2-9e28-db7d2f363e46 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:37.610970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f22a9d0-9d04-4cf2-9e28-db7d2f363e46 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:37.612548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 83/161] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:20:37.617629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d472183-6079-4754-b930-3798d27fc6da None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:37.618198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d472183-6079-4754-b930-3798d27fc6da None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:37.618662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d472183-6079-4754-b930-3798d27fc6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:37.635963 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2d472183-6079-4754-b930-3798d27fc6da None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:37.637610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2d472183-6079-4754-b930-3798d27fc6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:37.669695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:37.709927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 84/162] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:37.735792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d472183-6079-4754-b930-3798d27fc6da None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:37.809136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:37.837012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a9ac4-eac4-4129-9c62-7a1f357a2d04 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:37.908019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e369a-1bfc-4728-98bd-242c2c702d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:37.908919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e369a-1bfc-4728-98bd-242c2c702d39 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:37.909201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e369a-1bfc-4728-98bd-242c2c702d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:37.909478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e369a-1bfc-4728-98bd-242c2c702d39 None admin] PATH_INFO: `/v3/users/80cfaef5476c4d42b916f2cf9c155962` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:37.910738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e369a-1bfc-4728-98bd-242c2c702d39 None admin] RBAC: Authorizing `identity:get_user(user_id=80cfaef5476c4d42b916f2cf9c155962)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:37.912241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e369a-1bfc-4728-98bd-242c2c702d39 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:37.914198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 79/163] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:37 2022] GET /identity/v3/users/80cfaef5476c4d42b916f2cf9c155962 => generated 275 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:37.919616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e369a-1bfc-4728-98bd-242c2c702d39 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:38.002087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d472183-6079-4754-b930-3798d27fc6da None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:38.031922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:38.038967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e369a-1bfc-4728-98bd-242c2c702d39 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:38.062651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e31e44-f169-4cef-8e19-a967d53d0065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:38.063404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e31e44-f169-4cef-8e19-a967d53d0065 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:38.063688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e31e44-f169-4cef-8e19-a967d53d0065 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:38.063965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e31e44-f169-4cef-8e19-a967d53d0065 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:38.080384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e31e44-f169-4cef-8e19-a967d53d0065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:38.080648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91792dd6-0038-41eb-a7cc-9ca15675e734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:38.081419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91792dd6-0038-41eb-a7cc-9ca15675e734 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:38.081798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91792dd6-0038-41eb-a7cc-9ca15675e734 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:38.082184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91792dd6-0038-41eb-a7cc-9ca15675e734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d8e31e44-f169-4cef-8e19-a967d53d0065 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:38.093514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:38.094451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:38.095216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:38.101723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 85/164] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:20:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:38.107796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e31e44-f169-4cef-8e19-a967d53d0065 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:38.206690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:38.208514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91792dd6-0038-41eb-a7cc-9ca15675e734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:38.210067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91792dd6-0038-41eb-a7cc-9ca15675e734 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:38.230253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e31e44-f169-4cef-8e19-a967d53d0065 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:38.273906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53c7a-b552-4ac3-8e62-a63fa3ce8fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:38.274575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53c7a-b552-4ac3-8e62-a63fa3ce8fbf None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:38.274856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53c7a-b552-4ac3-8e62-a63fa3ce8fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:38.275126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53c7a-b552-4ac3-8e62-a63fa3ce8fbf None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:38.275935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b53c7a-b552-4ac3-8e62-a63fa3ce8fbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:38.276242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b53c7a-b552-4ac3-8e62-a63fa3ce8fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:38.277653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b53c7a-b552-4ac3-8e62-a63fa3ce8fbf None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:38.285953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 86/165] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:38.291689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53c7a-b552-4ac3-8e62-a63fa3ce8fbf None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:38.399643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53c7a-b552-4ac3-8e62-a63fa3ce8fbf None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:38.435918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7bcc-103f-458b-91a4-391a18454c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:38.437062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7bcc-103f-458b-91a4-391a18454c73 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:38.437480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7bcc-103f-458b-91a4-391a18454c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:38.437887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7bcc-103f-458b-91a4-391a18454c73 None admin] PATH_INFO: `/v3/users/c350860378554603affd063cfacfc453` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:38.439421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a7bcc-103f-458b-91a4-391a18454c73 None admin] RBAC: Authorizing `identity:get_user(user_id=c350860378554603affd063cfacfc453)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:38.441029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a7bcc-103f-458b-91a4-391a18454c73 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:38.442806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 87/166] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:38 2022] GET /identity/v3/users/c350860378554603affd063cfacfc453 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:38.449094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7bcc-103f-458b-91a4-391a18454c73 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:38.467766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:38.470236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:38.483785 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 80/167] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 02 03:20:38.486078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91792dd6-0038-41eb-a7cc-9ca15675e734 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:38.628232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7bcc-103f-458b-91a4-391a18454c73 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:38.689751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91792dd6-0038-41eb-a7cc-9ca15675e734 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:38.708729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29e072-c23e-4288-9049-db166a184de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:38.709728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29e072-c23e-4288-9049-db166a184de3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:38.710229 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29e072-c23e-4288-9049-db166a184de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:38.710970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc29e072-c23e-4288-9049-db166a184de3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:38.758776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8779bb2-9f5c-4ddf-8afe-d8abd25bc6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:38.760081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8779bb2-9f5c-4ddf-8afe-d8abd25bc6ff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:38.760745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8779bb2-9f5c-4ddf-8afe-d8abd25bc6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:38.761311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8779bb2-9f5c-4ddf-8afe-d8abd25bc6ff None admin] PATH_INFO: `/v3/projects/bade203bac76419a9e736d5b2e23806b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:38.763509 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8779bb2-9f5c-4ddf-8afe-d8abd25bc6ff None admin] RBAC: Authorizing `identity:get_project(project_id=bade203bac76419a9e736d5b2e23806b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:38.765210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8779bb2-9f5c-4ddf-8afe-d8abd25bc6ff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:38.771730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 81/168] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:38 2022] GET /identity/v3/projects/bade203bac76419a9e736d5b2e23806b => generated 312 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:38.773810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8779bb2-9f5c-4ddf-8afe-d8abd25bc6ff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:38.865726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:38.898484 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29e072-c23e-4288-9049-db166a184de3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:38.901947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc29e072-c23e-4288-9049-db166a184de3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:38.963887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:38.980904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8779bb2-9f5c-4ddf-8afe-d8abd25bc6ff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:39.074025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a5219-7510-4002-b564-8a27880edf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:39.074700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a5219-7510-4002-b564-8a27880edf8d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.074982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a5219-7510-4002-b564-8a27880edf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.075259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a5219-7510-4002-b564-8a27880edf8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.076141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a5219-7510-4002-b564-8a27880edf8d None admin] RBAC: Adding query filter params (role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=bade203bac76419a9e736d5b2e23806b, user.id=80cfaef5476c4d42b916f2cf9c155962) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:39.076454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a5219-7510-4002-b564-8a27880edf8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:39.080160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a5219-7510-4002-b564-8a27880edf8d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:39.092626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 82/169] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:38 2022] GET /identity/v3/role_assignments?user.id=80cfaef5476c4d42b916f2cf9c155962&scope.project.id=bade203bac76419a9e736d5b2e23806b&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:39.098353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516749d-b756-4a22-8970-742193e0c606 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.098811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516749d-b756-4a22-8970-742193e0c606 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.099256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516749d-b756-4a22-8970-742193e0c606 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.103504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 83/170] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:39.112765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca6847-f07f-4ef6-8b80-b0051fe480bb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.113238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca6847-f07f-4ef6-8b80-b0051fe480bb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.113684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca6847-f07f-4ef6-8b80-b0051fe480bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.145662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-acca6847-f07f-4ef6-8b80-b0051fe480bb None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:39.147265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-acca6847-f07f-4ef6-8b80-b0051fe480bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:39.172236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 88/171] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 02 03:20:39.177554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29e072-c23e-4288-9049-db166a184de3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:39.317490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 84/172] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:39.349021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca6847-f07f-4ef6-8b80-b0051fe480bb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:39.447391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc29e072-c23e-4288-9049-db166a184de3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:39.470180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:39.476656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:39.518924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d00f0-ec2b-4485-a5a1-75f82b82b9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:39.519834 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d00f0-ec2b-4485-a5a1-75f82b82b9bc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.520123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d00f0-ec2b-4485-a5a1-75f82b82b9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.520399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d00f0-ec2b-4485-a5a1-75f82b82b9bc None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.521854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d00f0-ec2b-4485-a5a1-75f82b82b9bc None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:39.523359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d00f0-ec2b-4485-a5a1-75f82b82b9bc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:39.525288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 89/173] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:39 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:39.532829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d00f0-ec2b-4485-a5a1-75f82b82b9bc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:39.562197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca6847-f07f-4ef6-8b80-b0051fe480bb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:39.632481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb3d5c-a9d0-4e7f-b2a7-5f28bf9d3fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:39.633234 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3d5c-a9d0-4e7f-b2a7-5f28bf9d3fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.633521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3d5c-a9d0-4e7f-b2a7-5f28bf9d3fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.633798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb3d5c-a9d0-4e7f-b2a7-5f28bf9d3fc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.657150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb3d5c-a9d0-4e7f-b2a7-5f28bf9d3fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b7eb3d5c-a9d0-4e7f-b2a7-5f28bf9d3fc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:39.668592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:39.669506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:39.670266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 85/174] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:20:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:39.671409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efacca-fe75-44e9-afc9-ec0c7ef4f7b2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.674487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efacca-fe75-44e9-afc9-ec0c7ef4f7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.675911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efacca-fe75-44e9-afc9-ec0c7ef4f7b2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.685301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 86/175] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:39 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:39.686160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efacca-fe75-44e9-afc9-ec0c7ef4f7b2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:39.736606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d00f0-ec2b-4485-a5a1-75f82b82b9bc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:39.828538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43552e14-289e-46df-9ff9-e19741054bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:39.829218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43552e14-289e-46df-9ff9-e19741054bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.829503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43552e14-289e-46df-9ff9-e19741054bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.829781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43552e14-289e-46df-9ff9-e19741054bf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.830633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43552e14-289e-46df-9ff9-e19741054bf5 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=94c3ce30be2e489e9210838c8132fd92, user.id=c350860378554603affd063cfacfc453) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:39.830949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43552e14-289e-46df-9ff9-e19741054bf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:39.832661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efacca-fe75-44e9-afc9-ec0c7ef4f7b2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:39.838326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43552e14-289e-46df-9ff9-e19741054bf5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:39.851141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:39.854170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 90/176] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:39 2022] GET /identity/v3/role_assignments?user.id=c350860378554603affd063cfacfc453&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:39.855883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817ef2-e629-4917-8778-aa5a99b9af1a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.857944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817ef2-e629-4917-8778-aa5a99b9af1a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.858841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc817ef2-e629-4917-8778-aa5a99b9af1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.872051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7928ebb-927c-4eb7-89be-4a863e973e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:39.872781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7928ebb-927c-4eb7-89be-4a863e973e70 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.873072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7928ebb-927c-4eb7-89be-4a863e973e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.873351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7928ebb-927c-4eb7-89be-4a863e973e70 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.874160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7928ebb-927c-4eb7-89be-4a863e973e70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:39.874475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7928ebb-927c-4eb7-89be-4a863e973e70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:39.876082 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7928ebb-927c-4eb7-89be-4a863e973e70 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:39.887009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 87/177] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:20:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:39.891638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612424e-ac38-4965-a200-52d4d6b7ba2f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:39.891638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612424e-ac38-4965-a200-52d4d6b7ba2f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:39.891638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612424e-ac38-4965-a200-52d4d6b7ba2f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:39.891638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 88/178] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:39.895685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612424e-ac38-4965-a200-52d4d6b7ba2f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:39.928536 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fc817ef2-e629-4917-8778-aa5a99b9af1a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:39.936946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fc817ef2-e629-4917-8778-aa5a99b9af1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:40.024875 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612424e-ac38-4965-a200-52d4d6b7ba2f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:40.034011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:40.067237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ef10e-6378-4dac-9bdb-444c1d056ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:40.068465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ef10e-6378-4dac-9bdb-444c1d056ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.068894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ef10e-6378-4dac-9bdb-444c1d056ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.069305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ef10e-6378-4dac-9bdb-444c1d056ff7 None admin] PATH_INFO: `/v3/users/80cfaef5476c4d42b916f2cf9c155962` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.070742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ef10e-6378-4dac-9bdb-444c1d056ff7 None admin] RBAC: Authorizing `identity:get_user(user_id=80cfaef5476c4d42b916f2cf9c155962)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:40.070866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 91/179] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:40.072373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7913b-aa14-4459-ae1f-3458982d52d3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.072664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7913b-aa14-4459-ae1f-3458982d52d3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.072977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ef10e-6378-4dac-9bdb-444c1d056ff7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:40.073102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7913b-aa14-4459-ae1f-3458982d52d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.075207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 89/180] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:39 2022] GET /identity/v3/users/80cfaef5476c4d42b916f2cf9c155962 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:40.083316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ef10e-6378-4dac-9bdb-444c1d056ff7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:40.090179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-28f7913b-aa14-4459-ae1f-3458982d52d3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:40.091371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-28f7913b-aa14-4459-ae1f-3458982d52d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:40.156713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:40.187469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 92/181] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:40.188936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30193e-05df-4e2c-8944-b91d1a80efca None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.189230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30193e-05df-4e2c-8944-b91d1a80efca None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.189512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30193e-05df-4e2c-8944-b91d1a80efca None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.190854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 93/182] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:40.191537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30193e-05df-4e2c-8944-b91d1a80efca None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:40.231002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ef10e-6378-4dac-9bdb-444c1d056ff7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:40.265922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd56b-97d1-464a-9383-96bd00f1c048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:40.266579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd56b-97d1-464a-9383-96bd00f1c048 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.266861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd56b-97d1-464a-9383-96bd00f1c048 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.267136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0dd56b-97d1-464a-9383-96bd00f1c048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.295701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30193e-05df-4e2c-8944-b91d1a80efca None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:40.302731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:40.331044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d362b-c9f9-498a-b6d9-8ce1dfecb95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:40.331867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d362b-c9f9-498a-b6d9-8ce1dfecb95d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.332141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d362b-c9f9-498a-b6d9-8ce1dfecb95d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.332410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d362b-c9f9-498a-b6d9-8ce1dfecb95d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.333154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dd56b-97d1-464a-9383-96bd00f1c048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:40.334616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0dd56b-97d1-464a-9383-96bd00f1c048 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:40.342464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d362b-c9f9-498a-b6d9-8ce1dfecb95d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-321d362b-c9f9-498a-b6d9-8ce1dfecb95d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:40.349980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:40.350831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:40.351667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 94/183] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:20:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:40.354258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918162d-bf97-4712-a484-5bebfc5e23ae None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.354561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918162d-bf97-4712-a484-5bebfc5e23ae None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.354845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918162d-bf97-4712-a484-5bebfc5e23ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.371752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0918162d-bf97-4712-a484-5bebfc5e23ae None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:40.372938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0918162d-bf97-4712-a484-5bebfc5e23ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:40.380569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:40.434541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 90/184] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:40.435411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd56b-97d1-464a-9383-96bd00f1c048 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:40.443330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 95/185] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:40.444646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99baea-edc9-4148-9077-e12c85893413 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.444934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99baea-edc9-4148-9077-e12c85893413 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.445209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99baea-edc9-4148-9077-e12c85893413 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.446628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 96/186] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:40.447169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99baea-edc9-4148-9077-e12c85893413 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:40.544977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:40.549223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0dd56b-97d1-464a-9383-96bd00f1c048 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:40.552036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99baea-edc9-4148-9077-e12c85893413 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:40.583031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c4a55-8068-4f65-a7d0-aa07448fb513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:40.583898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c4a55-8068-4f65-a7d0-aa07448fb513 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.584175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c4a55-8068-4f65-a7d0-aa07448fb513 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.584449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c4a55-8068-4f65-a7d0-aa07448fb513 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.584985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc879b-da7d-44ca-996a-0a884cb6f366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:40.585648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc879b-da7d-44ca-996a-0a884cb6f366 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.585931 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc879b-da7d-44ca-996a-0a884cb6f366 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.586208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc879b-da7d-44ca-996a-0a884cb6f366 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.586960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc879b-da7d-44ca-996a-0a884cb6f366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:40.587264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc879b-da7d-44ca-996a-0a884cb6f366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:40.588766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc879b-da7d-44ca-996a-0a884cb6f366 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:40.594488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c4a55-8068-4f65-a7d0-aa07448fb513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:40.596928 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 97/187] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:40.597743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc879b-da7d-44ca-996a-0a884cb6f366 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-020c4a55-8068-4f65-a7d0-aa07448fb513 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:40.606643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:40.607753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 91/188] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:20:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:40.613444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451b019-f3d5-4e18-830f-bd84645aa711 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.614139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451b019-f3d5-4e18-830f-bd84645aa711 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.614139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451b019-f3d5-4e18-830f-bd84645aa711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.621438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:40.627505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a451b019-f3d5-4e18-830f-bd84645aa711 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:40.628698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a451b019-f3d5-4e18-830f-bd84645aa711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:40.697448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 92/189] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:40.698554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451b019-f3d5-4e18-830f-bd84645aa711 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:40.704142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc879b-da7d-44ca-996a-0a884cb6f366 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:40.740780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e25e0-9bd3-440c-8eaa-b64bcf3a623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:40.741634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e25e0-9bd3-440c-8eaa-b64bcf3a623a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.741911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e25e0-9bd3-440c-8eaa-b64bcf3a623a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.742177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e25e0-9bd3-440c-8eaa-b64bcf3a623a None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.743397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e25e0-9bd3-440c-8eaa-b64bcf3a623a None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:40.745057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e25e0-9bd3-440c-8eaa-b64bcf3a623a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:40.746805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 98/190] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:40 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:40.747582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e25e0-9bd3-440c-8eaa-b64bcf3a623a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:40.774830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:40.799271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451b019-f3d5-4e18-830f-bd84645aa711 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:40.832444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ddc1f-297e-4662-883a-cb26e5bf39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:40.833238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ddc1f-297e-4662-883a-cb26e5bf39fe None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.833516 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ddc1f-297e-4662-883a-cb26e5bf39fe None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.833788 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ddc1f-297e-4662-883a-cb26e5bf39fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.836381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:40.845577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ddc1f-297e-4662-883a-cb26e5bf39fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:40.848790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e25e0-9bd3-440c-8eaa-b64bcf3a623a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-299ddc1f-297e-4662-883a-cb26e5bf39fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:40.853270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:40.854167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:40.854938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 93/191] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:40.855619 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ddc1f-297e-4662-883a-cb26e5bf39fe None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:40.884319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dda8ec-d5cc-42b1-b99f-cf2e2be34680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:40.885195 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dda8ec-d5cc-42b1-b99f-cf2e2be34680 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.885543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dda8ec-d5cc-42b1-b99f-cf2e2be34680 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.885819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dda8ec-d5cc-42b1-b99f-cf2e2be34680 None admin] PATH_INFO: `/v3/users/11d0038b8824424b814f6ab7ab0f3d0c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.899606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dda8ec-d5cc-42b1-b99f-cf2e2be34680 None admin] RBAC: Authorizing `identity:get_user(user_id=11d0038b8824424b814f6ab7ab0f3d0c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:40.901166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dda8ec-d5cc-42b1-b99f-cf2e2be34680 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:40.902852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 99/192] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:40 2022] GET /identity/v3/users/11d0038b8824424b814f6ab7ab0f3d0c => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:40.908001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dda8ec-d5cc-42b1-b99f-cf2e2be34680 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:40.956396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ddc1f-297e-4662-883a-cb26e5bf39fe None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:40.990502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe146e8-dcc0-4d8f-ada5-3f930a6e61e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:40.991167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe146e8-dcc0-4d8f-ada5-3f930a6e61e7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:40.991476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe146e8-dcc0-4d8f-ada5-3f930a6e61e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:40.991606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:40.991744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe146e8-dcc0-4d8f-ada5-3f930a6e61e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:40.992541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe146e8-dcc0-4d8f-ada5-3f930a6e61e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:40.992895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe146e8-dcc0-4d8f-ada5-3f930a6e61e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:40.994397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe146e8-dcc0-4d8f-ada5-3f930a6e61e7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.007443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 94/193] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:20:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:41.007443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe146e8-dcc0-4d8f-ada5-3f930a6e61e7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:41.017184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dda8ec-d5cc-42b1-b99f-cf2e2be34680 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:41.051793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24315525-bb5c-429b-893d-2e8439b53d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:41.052587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24315525-bb5c-429b-893d-2e8439b53d97 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:41.052867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24315525-bb5c-429b-893d-2e8439b53d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:41.053142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24315525-bb5c-429b-893d-2e8439b53d97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:41.054332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24315525-bb5c-429b-893d-2e8439b53d97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:41.055839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24315525-bb5c-429b-893d-2e8439b53d97 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.057829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 100/194] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:41.058695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24315525-bb5c-429b-893d-2e8439b53d97 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:41.066666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:41.110759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe146e8-dcc0-4d8f-ada5-3f930a6e61e7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:41.148367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773045c2-d043-4b31-85de-eb37cd6cad0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:41.149021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773045c2-d043-4b31-85de-eb37cd6cad0a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:41.149303 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773045c2-d043-4b31-85de-eb37cd6cad0a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:41.149576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773045c2-d043-4b31-85de-eb37cd6cad0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:41.150405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773045c2-d043-4b31-85de-eb37cd6cad0a None admin] RBAC: Adding query filter params (role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=94c3ce30be2e489e9210838c8132fd92, user.id=80cfaef5476c4d42b916f2cf9c155962) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:41.150712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773045c2-d043-4b31-85de-eb37cd6cad0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:41.152476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773045c2-d043-4b31-85de-eb37cd6cad0a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.158453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 95/195] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:41 2022] GET /identity/v3/role_assignments?user.id=80cfaef5476c4d42b916f2cf9c155962&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=62ac4326d6d742c5b6684746953362b1 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:41.158985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773045c2-d043-4b31-85de-eb37cd6cad0a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:41.170236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24315525-bb5c-429b-893d-2e8439b53d97 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:41.208530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df46d-9c78-4f87-b317-dffeef622890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:41.209200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df46d-9c78-4f87-b317-dffeef622890 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:41.209567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df46d-9c78-4f87-b317-dffeef622890 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:41.209852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df46d-9c78-4f87-b317-dffeef622890 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:41.210629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2df46d-9c78-4f87-b317-dffeef622890 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:41.210955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2df46d-9c78-4f87-b317-dffeef622890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:41.212414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2df46d-9c78-4f87-b317-dffeef622890 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.226003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 101/196] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:41.226003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df46d-9c78-4f87-b317-dffeef622890 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:41.240548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:41.279415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773045c2-d043-4b31-85de-eb37cd6cad0a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:41.304241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:41.317777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cece9-b644-4359-8658-1aebdb9e80d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:41.318444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cece9-b644-4359-8658-1aebdb9e80d0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:41.318724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cece9-b644-4359-8658-1aebdb9e80d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:41.318997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cece9-b644-4359-8658-1aebdb9e80d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:41.334145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df46d-9c78-4f87-b317-dffeef622890 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:41.379037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc63a29-6d82-4416-a75d-81a64f18c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:41.380780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc63a29-6d82-4416-a75d-81a64f18c612 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:41.381316 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc63a29-6d82-4416-a75d-81a64f18c612 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:41.382104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc63a29-6d82-4416-a75d-81a64f18c612 None admin] PATH_INFO: `/v3/users/11d0038b8824424b814f6ab7ab0f3d0c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:41.386115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc63a29-6d82-4416-a75d-81a64f18c612 None admin] RBAC: Authorizing `identity:get_user(user_id=11d0038b8824424b814f6ab7ab0f3d0c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:41.386115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc63a29-6d82-4416-a75d-81a64f18c612 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.387823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 102/197] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:41 2022] GET /identity/v3/users/11d0038b8824424b814f6ab7ab0f3d0c => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:41.388825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc63a29-6d82-4416-a75d-81a64f18c612 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:41.412407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cece9-b644-4359-8658-1aebdb9e80d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:41.414321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cece9-b644-4359-8658-1aebdb9e80d0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.480693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:41.500970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc63a29-6d82-4416-a75d-81a64f18c612 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:41.525691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 96/198] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:41.526989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cece9-b644-4359-8658-1aebdb9e80d0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:41.543958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913266c-d1a2-4369-acff-fe9428154436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:41.544637 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913266c-d1a2-4369-acff-fe9428154436 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:41.544929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913266c-d1a2-4369-acff-fe9428154436 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:41.545208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913266c-d1a2-4369-acff-fe9428154436 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:41.545922 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913266c-d1a2-4369-acff-fe9428154436 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:41.547418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7913266c-d1a2-4369-acff-fe9428154436 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.563036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:41.568198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 103/199] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:20:41 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:41.569029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913266c-d1a2-4369-acff-fe9428154436 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:41.675972 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cece9-b644-4359-8658-1aebdb9e80d0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:41.701767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7913266c-d1a2-4369-acff-fe9428154436 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:41.718725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09af8d-e54f-48e2-a424-2db0225709b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:41.719644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09af8d-e54f-48e2-a424-2db0225709b7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:41.719930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09af8d-e54f-48e2-a424-2db0225709b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:41.720202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09af8d-e54f-48e2-a424-2db0225709b7 None admin] PATH_INFO: `/v3/users/94585aa232a74c0584bdb9b5b044ebd7` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:41.721469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09af8d-e54f-48e2-a424-2db0225709b7 None admin] RBAC: Authorizing `identity:get_user(user_id=94585aa232a74c0584bdb9b5b044ebd7)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:41.723121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09af8d-e54f-48e2-a424-2db0225709b7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.724953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 97/200] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:41 2022] GET /identity/v3/users/94585aa232a74c0584bdb9b5b044ebd7 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:41.725736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09af8d-e54f-48e2-a424-2db0225709b7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:41.752149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09fb30-07b2-4917-9327-52762ff5ab5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:41.753323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09fb30-07b2-4917-9327-52762ff5ab5d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:41.753608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09fb30-07b2-4917-9327-52762ff5ab5d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:41.753879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09fb30-07b2-4917-9327-52762ff5ab5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:41.764388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:41.845997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:41.846787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09fb30-07b2-4917-9327-52762ff5ab5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:41.848856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09fb30-07b2-4917-9327-52762ff5ab5d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.857599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09af8d-e54f-48e2-a424-2db0225709b7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:41.910921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a953-7a6d-473e-b993-10b556dbeaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:41.911828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a953-7a6d-473e-b993-10b556dbeaff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:41.912232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a953-7a6d-473e-b993-10b556dbeaff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:41.912508 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a953-7a6d-473e-b993-10b556dbeaff None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:41.914956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a953-7a6d-473e-b993-10b556dbeaff None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:41.916997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a953-7a6d-473e-b993-10b556dbeaff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:41.918964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 98/201] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:41 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:41.919969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a953-7a6d-473e-b993-10b556dbeaff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:41.986541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 104/202] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:41.987524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09fb30-07b2-4917-9327-52762ff5ab5d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:42.047017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:42.058592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a953-7a6d-473e-b993-10b556dbeaff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:42.108110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496558fb-7764-4bd5-a301-98cf03b3855d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:42.108604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496558fb-7764-4bd5-a301-98cf03b3855d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:42.108886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496558fb-7764-4bd5-a301-98cf03b3855d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:42.109153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496558fb-7764-4bd5-a301-98cf03b3855d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:42.133528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09fb30-07b2-4917-9327-52762ff5ab5d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:42.144353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:42.177828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf829a-cbe6-4ab8-8f77-e59a3d5257ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:42.178643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf829a-cbe6-4ab8-8f77-e59a3d5257ea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:42.178928 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf829a-cbe6-4ab8-8f77-e59a3d5257ea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:42.179240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf829a-cbe6-4ab8-8f77-e59a3d5257ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:42.180453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf829a-cbe6-4ab8-8f77-e59a3d5257ea None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=4b61245437cd4d14997cf3df4ea706f0, user.id=11d0038b8824424b814f6ab7ab0f3d0c) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:42.181332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf829a-cbe6-4ab8-8f77-e59a3d5257ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:42.182884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf829a-cbe6-4ab8-8f77-e59a3d5257ea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:42.189695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 105/203] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:41 2022] GET /identity/v3/role_assignments?user.id=11d0038b8824424b814f6ab7ab0f3d0c&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:42.190608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf829a-cbe6-4ab8-8f77-e59a3d5257ea None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:42.206752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496558fb-7764-4bd5-a301-98cf03b3855d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:42.208292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496558fb-7764-4bd5-a301-98cf03b3855d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:42.345278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf829a-cbe6-4ab8-8f77-e59a3d5257ea None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:42.348056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:42.418531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821082f3-baee-4e89-ba08-53a8dd1f49ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:42.419664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821082f3-baee-4e89-ba08-53a8dd1f49ff None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:42.419947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821082f3-baee-4e89-ba08-53a8dd1f49ff None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:42.420220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821082f3-baee-4e89-ba08-53a8dd1f49ff None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:42.423227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821082f3-baee-4e89-ba08-53a8dd1f49ff None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:42.423825 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 99/204] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:42.424840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821082f3-baee-4e89-ba08-53a8dd1f49ff None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:42.426846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 106/205] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:42 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:42.433128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496558fb-7764-4bd5-a301-98cf03b3855d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:42.434526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821082f3-baee-4e89-ba08-53a8dd1f49ff None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:42.515668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:42.588444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496558fb-7764-4bd5-a301-98cf03b3855d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:42.601192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821082f3-baee-4e89-ba08-53a8dd1f49ff None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:42.636735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981069cb-a5b6-482d-a760-15d6cb38b76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:42.637637 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981069cb-a5b6-482d-a760-15d6cb38b76e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:42.638055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981069cb-a5b6-482d-a760-15d6cb38b76e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:42.638489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981069cb-a5b6-482d-a760-15d6cb38b76e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:42.639495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981069cb-a5b6-482d-a760-15d6cb38b76e None admin] RBAC: Adding query filter params (role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=4b61245437cd4d14997cf3df4ea706f0, user.id=11d0038b8824424b814f6ab7ab0f3d0c) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:42.639939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981069cb-a5b6-482d-a760-15d6cb38b76e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:42.641524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981069cb-a5b6-482d-a760-15d6cb38b76e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:42.647428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 107/206] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:42 2022] GET /identity/v3/role_assignments?user.id=11d0038b8824424b814f6ab7ab0f3d0c&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=62ac4326d6d742c5b6684746953362b1 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:42.656514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28671793-178d-4ac7-b3f3-923fc735364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:42.657664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28671793-178d-4ac7-b3f3-923fc735364a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:42.658081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28671793-178d-4ac7-b3f3-923fc735364a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:42.658496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28671793-178d-4ac7-b3f3-923fc735364a None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:42.668366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28671793-178d-4ac7-b3f3-923fc735364a None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:42.670186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28671793-178d-4ac7-b3f3-923fc735364a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:42.672360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 100/207] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:42 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:42.678498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981069cb-a5b6-482d-a760-15d6cb38b76e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:42.719000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:42.790736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981069cb-a5b6-482d-a760-15d6cb38b76e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:42.828001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b339e99-5de9-4860-946d-2677012a8d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:42.828899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b339e99-5de9-4860-946d-2677012a8d96 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:42.829319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b339e99-5de9-4860-946d-2677012a8d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:42.829765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b339e99-5de9-4860-946d-2677012a8d96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:42.830735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b339e99-5de9-4860-946d-2677012a8d96 None admin] RBAC: Adding query filter params (role.id=3ff68e8f3bb340bf985c55b19b58c222, scope.project.id=94c3ce30be2e489e9210838c8132fd92, user.id=94585aa232a74c0584bdb9b5b044ebd7) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:42.831194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b339e99-5de9-4860-946d-2677012a8d96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:42.832814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b339e99-5de9-4860-946d-2677012a8d96 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:42.840598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 108/208] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:42 2022] GET /identity/v3/role_assignments?user.id=94585aa232a74c0584bdb9b5b044ebd7&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=3ff68e8f3bb340bf985c55b19b58c222 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:43.978399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f8f1a-97ed-4b82-8077-2aa3daa9cf61 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:43.979371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f8f1a-97ed-4b82-8077-2aa3daa9cf61 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:43.979784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f8f1a-97ed-4b82-8077-2aa3daa9cf61 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:43.981558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 101/209] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:43.987131 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765346-76c2-4c9f-8822-131938861054 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:43.987959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765346-76c2-4c9f-8822-131938861054 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:43.988459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46765346-76c2-4c9f-8822-131938861054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.005952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-46765346-76c2-4c9f-8822-131938861054 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:44.007447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-46765346-76c2-4c9f-8822-131938861054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:44.058081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd2130-a641-4f8c-b50c-dff5e0ec20f1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.058600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd2130-a641-4f8c-b50c-dff5e0ec20f1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.058874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd2130-a641-4f8c-b50c-dff5e0ec20f1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.060427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 102/210] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:20:44.067430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b791e-0ced-4e26-9b46-a32b24ee2601 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.067430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b791e-0ced-4e26-9b46-a32b24ee2601 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.068014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b791e-0ced-4e26-9b46-a32b24ee2601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.092128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f94b791e-0ced-4e26-9b46-a32b24ee2601 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:44.093348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f94b791e-0ced-4e26-9b46-a32b24ee2601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:44.094276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 109/211] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:44.118076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765346-76c2-4c9f-8822-131938861054 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:44.150070 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:44.179084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 103/212] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:44.203490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b791e-0ced-4e26-9b46-a32b24ee2601 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:44.229215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46765346-76c2-4c9f-8822-131938861054 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:44.263446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:44.264284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ecea0-d9f8-4710-87c8-f8fcd45b9406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:44.265447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ecea0-d9f8-4710-87c8-f8fcd45b9406 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.266029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ecea0-d9f8-4710-87c8-f8fcd45b9406 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.266441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ecea0-d9f8-4710-87c8-f8fcd45b9406 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.277753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ecea0-d9f8-4710-87c8-f8fcd45b9406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b85ecea0-d9f8-4710-87c8-f8fcd45b9406 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:44.285552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:44.286626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:44.287614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:44.294553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 110/213] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:44.296016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ecea0-d9f8-4710-87c8-f8fcd45b9406 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:44.316029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b791e-0ced-4e26-9b46-a32b24ee2601 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:44.353049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db841f6d-7d6d-4e09-8f6a-a21db42ef564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:44.354171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db841f6d-7d6d-4e09-8f6a-a21db42ef564 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.355127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db841f6d-7d6d-4e09-8f6a-a21db42ef564 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.355552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db841f6d-7d6d-4e09-8f6a-a21db42ef564 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.356860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db841f6d-7d6d-4e09-8f6a-a21db42ef564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:44.358440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db841f6d-7d6d-4e09-8f6a-a21db42ef564 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:44.360267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 104/214] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:44.365934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db841f6d-7d6d-4e09-8f6a-a21db42ef564 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:44.394381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:44.408803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ecea0-d9f8-4710-87c8-f8fcd45b9406 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:44.444247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e24b74-6c2d-46f4-ad2e-27b216f83d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:44.445150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e24b74-6c2d-46f4-ad2e-27b216f83d10 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.445597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e24b74-6c2d-46f4-ad2e-27b216f83d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.446004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e24b74-6c2d-46f4-ad2e-27b216f83d10 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.446883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e24b74-6c2d-46f4-ad2e-27b216f83d10 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:44.447341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e24b74-6c2d-46f4-ad2e-27b216f83d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:44.449203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e24b74-6c2d-46f4-ad2e-27b216f83d10 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:44.462632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 111/215] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:44.464599 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e24b74-6c2d-46f4-ad2e-27b216f83d10 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:44.476547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db841f6d-7d6d-4e09-8f6a-a21db42ef564 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:44.503033 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:44.514087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d793f-3682-4569-b8a3-37fe0d58f7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:44.514771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d793f-3682-4569-b8a3-37fe0d58f7bf None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.515052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d793f-3682-4569-b8a3-37fe0d58f7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.515356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d793f-3682-4569-b8a3-37fe0d58f7bf None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.516071 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d793f-3682-4569-b8a3-37fe0d58f7bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:44.517537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d793f-3682-4569-b8a3-37fe0d58f7bf None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:44.538395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 105/216] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:20:44 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:20:44.538395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d839-0886-4ef7-82c3-222b1b37ab59 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.538395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d839-0886-4ef7-82c3-222b1b37ab59 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.538395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d839-0886-4ef7-82c3-222b1b37ab59 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.538395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 106/217] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:44.547766 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1db2c-f867-4c5c-8093-116f208141b3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.548144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1db2c-f867-4c5c-8093-116f208141b3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.548404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1db2c-f867-4c5c-8093-116f208141b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.574739 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9df1db2c-f867-4c5c-8093-116f208141b3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:44.576106 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9df1db2c-f867-4c5c-8093-116f208141b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:44.598779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e24b74-6c2d-46f4-ad2e-27b216f83d10 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:44.652301 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e027865-eefe-4909-b9a6-33b85e07dc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:44.653220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e027865-eefe-4909-b9a6-33b85e07dc1f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.653507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e027865-eefe-4909-b9a6-33b85e07dc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.653868 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e027865-eefe-4909-b9a6-33b85e07dc1f None admin] PATH_INFO: `/v3/users/d0b19bfb85b74fb0b4d118bb192bfb53` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.658227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 107/218] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:44.671615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e027865-eefe-4909-b9a6-33b85e07dc1f None admin] RBAC: Authorizing `identity:get_user(user_id=d0b19bfb85b74fb0b4d118bb192bfb53)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:44.673063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e027865-eefe-4909-b9a6-33b85e07dc1f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:44.675767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 112/219] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:44 2022] GET /identity/v3/users/d0b19bfb85b74fb0b4d118bb192bfb53 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:44.680469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1db2c-f867-4c5c-8093-116f208141b3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:44.681150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e027865-eefe-4909-b9a6-33b85e07dc1f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:44.693528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:44.812313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1db2c-f867-4c5c-8093-116f208141b3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:44.813349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e027865-eefe-4909-b9a6-33b85e07dc1f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:44.828812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:44.859430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02868c85-51e9-4bff-9350-98a71bc6970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:44.860518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868c85-51e9-4bff-9350-98a71bc6970e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.860949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868c85-51e9-4bff-9350-98a71bc6970e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.861301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868c85-51e9-4bff-9350-98a71bc6970e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.863944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d68e48-929f-42d5-ba13-de4e1679e5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:44.864677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d68e48-929f-42d5-ba13-de4e1679e5df None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.865020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d68e48-929f-42d5-ba13-de4e1679e5df None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.865275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d68e48-929f-42d5-ba13-de4e1679e5df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.874595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02868c85-51e9-4bff-9350-98a71bc6970e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-02868c85-51e9-4bff-9350-98a71bc6970e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:44.882553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:44.883532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:44.884304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 108/220] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:20:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:44.886080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd36d0-a843-4cc6-a0c9-ad18e59b5a3e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:44.886080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd36d0-a843-4cc6-a0c9-ad18e59b5a3e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:44.886351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd36d0-a843-4cc6-a0c9-ad18e59b5a3e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:44.887706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 109/221] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:44.892301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd36d0-a843-4cc6-a0c9-ad18e59b5a3e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:44.959591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d68e48-929f-42d5-ba13-de4e1679e5df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:44.961074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d68e48-929f-42d5-ba13-de4e1679e5df None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:44.971036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:45.023941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd36d0-a843-4cc6-a0c9-ad18e59b5a3e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:45.069925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157f999-25d2-486b-9a2a-1e246d5afdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:45.070947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157f999-25d2-486b-9a2a-1e246d5afdae None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.071526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157f999-25d2-486b-9a2a-1e246d5afdae None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.072009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157f999-25d2-486b-9a2a-1e246d5afdae None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.073550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157f999-25d2-486b-9a2a-1e246d5afdae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:45.074067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157f999-25d2-486b-9a2a-1e246d5afdae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:45.076159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157f999-25d2-486b-9a2a-1e246d5afdae None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:45.087773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 113/222] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:45.089347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 110/223] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:45.091530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254a444-36f3-4865-a287-1dfc01f001f5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.091979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254a444-36f3-4865-a287-1dfc01f001f5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.092381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254a444-36f3-4865-a287-1dfc01f001f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.094783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157f999-25d2-486b-9a2a-1e246d5afdae None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:45.120527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d254a444-36f3-4865-a287-1dfc01f001f5 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:45.121921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d254a444-36f3-4865-a287-1dfc01f001f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:45.131997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:45.197750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 114/224] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:45.199414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254a444-36f3-4865-a287-1dfc01f001f5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:45.223963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157f999-25d2-486b-9a2a-1e246d5afdae None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:45.246219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:45.271759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdb9cd-e7bd-4a8c-86f8-a3a311ec16c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:45.272902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdb9cd-e7bd-4a8c-86f8-a3a311ec16c3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.273188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdb9cd-e7bd-4a8c-86f8-a3a311ec16c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.273466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdb9cd-e7bd-4a8c-86f8-a3a311ec16c3 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.275307 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdb9cd-e7bd-4a8c-86f8-a3a311ec16c3 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:45.277267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdb9cd-e7bd-4a8c-86f8-a3a311ec16c3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:45.279246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 111/225] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:45 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:45.280037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdb9cd-e7bd-4a8c-86f8-a3a311ec16c3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:45.321677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254a444-36f3-4865-a287-1dfc01f001f5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:45.363580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc51a94-4e73-43b2-802b-248df104e5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:45.364989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc51a94-4e73-43b2-802b-248df104e5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.365476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc51a94-4e73-43b2-802b-248df104e5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.365989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc51a94-4e73-43b2-802b-248df104e5d5 None admin] PATH_INFO: `/v3/users/11d0038b8824424b814f6ab7ab0f3d0c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.367391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc51a94-4e73-43b2-802b-248df104e5d5 None admin] RBAC: Authorizing `identity:get_user(user_id=11d0038b8824424b814f6ab7ab0f3d0c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:45.369217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc51a94-4e73-43b2-802b-248df104e5d5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:45.375840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 115/226] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:45 2022] GET /identity/v3/users/11d0038b8824424b814f6ab7ab0f3d0c => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:45.376938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc51a94-4e73-43b2-802b-248df104e5d5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:45.385261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:45.388138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdb9cd-e7bd-4a8c-86f8-a3a311ec16c3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:45.425131 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a6e11-aaeb-47cc-a667-aaa44a28b02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:45.426028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a6e11-aaeb-47cc-a667-aaa44a28b02f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.426313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a6e11-aaeb-47cc-a667-aaa44a28b02f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.426591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a6e11-aaeb-47cc-a667-aaa44a28b02f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.437809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a6e11-aaeb-47cc-a667-aaa44a28b02f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-fc4a6e11-aaeb-47cc-a667-aaa44a28b02f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:45.446497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:45.447483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 112/227] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:20:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:45.449484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a6e11-aaeb-47cc-a667-aaa44a28b02f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:45.507888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc51a94-4e73-43b2-802b-248df104e5d5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:45.515998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:45.545738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5892c8-699d-40b4-88ad-999053870cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:45.546543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5892c8-699d-40b4-88ad-999053870cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.546949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5892c8-699d-40b4-88ad-999053870cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.547358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5892c8-699d-40b4-88ad-999053870cc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.548448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5892c8-699d-40b4-88ad-999053870cc6 None admin] RBAC: Adding query filter params (role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=4b61245437cd4d14997cf3df4ea706f0, user.id=d0b19bfb85b74fb0b4d118bb192bfb53) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:45.548662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5892c8-699d-40b4-88ad-999053870cc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:45.550199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5892c8-699d-40b4-88ad-999053870cc6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:45.556480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 116/228] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:45 2022] GET /identity/v3/role_assignments?user.id=d0b19bfb85b74fb0b4d118bb192bfb53&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:45.558011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5892c8-699d-40b4-88ad-999053870cc6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:45.558316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a6e11-aaeb-47cc-a667-aaa44a28b02f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:45.597114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5d5da-30d7-4d89-8132-31b5b68844cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:45.598318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5d5da-30d7-4d89-8132-31b5b68844cb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.598795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5d5da-30d7-4d89-8132-31b5b68844cb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.599235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5d5da-30d7-4d89-8132-31b5b68844cb None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.600635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5d5da-30d7-4d89-8132-31b5b68844cb None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:45.602361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5d5da-30d7-4d89-8132-31b5b68844cb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:45.604636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 113/229] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:45 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:45.619120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5d5da-30d7-4d89-8132-31b5b68844cb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:45.677101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5892c8-699d-40b4-88ad-999053870cc6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:45.699458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:45.720685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d8f37-e5a6-411d-81f0-d53fe3e6fac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:45.721748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d8f37-e5a6-411d-81f0-d53fe3e6fac3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.722165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d8f37-e5a6-411d-81f0-d53fe3e6fac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.722571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d8f37-e5a6-411d-81f0-d53fe3e6fac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.729051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d8f37-e5a6-411d-81f0-d53fe3e6fac3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:45.729051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d8f37-e5a6-411d-81f0-d53fe3e6fac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:45.729051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d8f37-e5a6-411d-81f0-d53fe3e6fac3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:45.740160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 117/230] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:20:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:45.745399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d8f37-e5a6-411d-81f0-d53fe3e6fac3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:45.783697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:45.854340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d8f37-e5a6-411d-81f0-d53fe3e6fac3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:45.856643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5d5da-30d7-4d89-8132-31b5b68844cb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:45.899690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be1ef-1db8-4040-84fd-0bf167e433bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:45.900595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be1ef-1db8-4040-84fd-0bf167e433bf None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.900909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be1ef-1db8-4040-84fd-0bf167e433bf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.901186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be1ef-1db8-4040-84fd-0bf167e433bf None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0/users/11d0038b8824424b814f6ab7ab0f3d0c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.903171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be1ef-1db8-4040-84fd-0bf167e433bf None admin] RBAC: Authorizing `identity:create_grant(project_id=4b61245437cd4d14997cf3df4ea706f0, user_id=11d0038b8824424b814f6ab7ab0f3d0c, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:45.904991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be1ef-1db8-4040-84fd-0bf167e433bf None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:45.922833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 114/231] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:20:45 2022] PUT /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0/users/11d0038b8824424b814f6ab7ab0f3d0c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:20:45.925093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02516c76-cf84-48d3-a9ae-6ca265ec111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:45.926163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02516c76-cf84-48d3-a9ae-6ca265ec111e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:45.926589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02516c76-cf84-48d3-a9ae-6ca265ec111e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:45.927037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02516c76-cf84-48d3-a9ae-6ca265ec111e None admin] PATH_INFO: `/v3/users/11d0038b8824424b814f6ab7ab0f3d0c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:45.928545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02516c76-cf84-48d3-a9ae-6ca265ec111e None admin] RBAC: Authorizing `identity:get_user(user_id=11d0038b8824424b814f6ab7ab0f3d0c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:45.930256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02516c76-cf84-48d3-a9ae-6ca265ec111e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:45.933243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 118/232] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:45 2022] GET /identity/v3/users/11d0038b8824424b814f6ab7ab0f3d0c => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:45.938898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be1ef-1db8-4040-84fd-0bf167e433bf None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:46.035846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:46.050395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be1ef-1db8-4040-84fd-0bf167e433bf None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:46.086929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f33247-0683-407a-8ff9-7431a303274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:46.087854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f33247-0683-407a-8ff9-7431a303274e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:46.088135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f33247-0683-407a-8ff9-7431a303274e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:46.088476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f33247-0683-407a-8ff9-7431a303274e None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:46.089741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f33247-0683-407a-8ff9-7431a303274e None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:46.091211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f33247-0683-407a-8ff9-7431a303274e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:46.093259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 115/233] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:45 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:46.099068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02516c76-cf84-48d3-a9ae-6ca265ec111e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:46.202384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:46.205480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02516c76-cf84-48d3-a9ae-6ca265ec111e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:46.239082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbbf7d-5246-414d-9eb8-83973cb1497d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:46.240011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbbf7d-5246-414d-9eb8-83973cb1497d None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:46.240296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbbf7d-5246-414d-9eb8-83973cb1497d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:46.240570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbbf7d-5246-414d-9eb8-83973cb1497d None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0/users/11d0038b8824424b814f6ab7ab0f3d0c/roles/62ac4326d6d742c5b6684746953362b1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:46.242821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbbf7d-5246-414d-9eb8-83973cb1497d None admin] RBAC: Authorizing `identity:create_grant(project_id=4b61245437cd4d14997cf3df4ea706f0, user_id=11d0038b8824424b814f6ab7ab0f3d0c, role_id=62ac4326d6d742c5b6684746953362b1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:46.244316 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbbf7d-5246-414d-9eb8-83973cb1497d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:46.265955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 119/234] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:20:46 2022] PUT /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0/users/11d0038b8824424b814f6ab7ab0f3d0c/roles/62ac4326d6d742c5b6684746953362b1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:20:47.103003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8642d3-96c8-4112-94da-79efef4db3c3 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:47.103585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8642d3-96c8-4112-94da-79efef4db3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:47.103585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8642d3-96c8-4112-94da-79efef4db3c3 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:47.105468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 116/235] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:47.110592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbe26f-37d5-4f66-8ddd-0edfdc3fc17d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:47.110983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbe26f-37d5-4f66-8ddd-0edfdc3fc17d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:47.111141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbe26f-37d5-4f66-8ddd-0edfdc3fc17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:47.130216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a3466-2a2e-4fe0-9754-cc7882ee8959 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:47.130820 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a3466-2a2e-4fe0-9754-cc7882ee8959 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:47.131287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a3466-2a2e-4fe0-9754-cc7882ee8959 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:47.133273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 117/236] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:47.133462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-34dbe26f-37d5-4f66-8ddd-0edfdc3fc17d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:47.134654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-34dbe26f-37d5-4f66-8ddd-0edfdc3fc17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:47.140467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f091a-3fcf-4974-99f4-bee4f46e684e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:47.140938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f091a-3fcf-4974-99f4-bee4f46e684e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:47.141418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f091a-3fcf-4974-99f4-bee4f46e684e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:47.176348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-400f091a-3fcf-4974-99f4-bee4f46e684e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:47.177553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-400f091a-3fcf-4974-99f4-bee4f46e684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:47.261276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 120/237] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:47.270669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 118/238] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:47.286162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbe26f-37d5-4f66-8ddd-0edfdc3fc17d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:47.300177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f091a-3fcf-4974-99f4-bee4f46e684e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:47.392536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:47.409797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f091a-3fcf-4974-99f4-bee4f46e684e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:47.447652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2c0fd-9549-4542-8fea-705294c7c69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:47.448750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2c0fd-9549-4542-8fea-705294c7c69b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:47.449611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2c0fd-9549-4542-8fea-705294c7c69b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:47.450063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e2c0fd-9549-4542-8fea-705294c7c69b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:47.451752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2c0fd-9549-4542-8fea-705294c7c69b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:47.453388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e2c0fd-9549-4542-8fea-705294c7c69b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:47.455513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 119/239] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:47.461344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2c0fd-9549-4542-8fea-705294c7c69b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:47.491665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:47.509407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbe26f-37d5-4f66-8ddd-0edfdc3fc17d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:47.571168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e2c0fd-9549-4542-8fea-705294c7c69b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:47.587458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27de0f-8a6e-4e78-a365-193ad6a2ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:47.588411 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27de0f-8a6e-4e78-a365-193ad6a2ca52 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:47.588696 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27de0f-8a6e-4e78-a365-193ad6a2ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:47.588973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27de0f-8a6e-4e78-a365-193ad6a2ca52 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:47.609446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563d1c0-2a13-4382-8074-1902221b2c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:47.610361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563d1c0-2a13-4382-8074-1902221b2c10 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:47.610814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563d1c0-2a13-4382-8074-1902221b2c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:47.611232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563d1c0-2a13-4382-8074-1902221b2c10 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:47.612112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563d1c0-2a13-4382-8074-1902221b2c10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:47.613723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563d1c0-2a13-4382-8074-1902221b2c10 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:47.628575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27de0f-8a6e-4e78-a365-193ad6a2ca52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ce27de0f-8a6e-4e78-a365-193ad6a2ca52 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:47.637714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:47.638670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:47.639476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:47.642006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 121/240] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:47.648231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27de0f-8a6e-4e78-a365-193ad6a2ca52 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:47.648397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 120/241] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:20:47 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:20:47.819154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27de0f-8a6e-4e78-a365-193ad6a2ca52 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:47.862115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42577a-7680-42eb-9769-0831459a02de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:47.862807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42577a-7680-42eb-9769-0831459a02de None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:47.863089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42577a-7680-42eb-9769-0831459a02de None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:47.863388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42577a-7680-42eb-9769-0831459a02de None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:47.864184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42577a-7680-42eb-9769-0831459a02de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:47.864449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42577a-7680-42eb-9769-0831459a02de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:47.865847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42577a-7680-42eb-9769-0831459a02de None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:47.878765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 122/242] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:47.884287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563d1c0-2a13-4382-8074-1902221b2c10 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:47.900172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:47.993495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563d1c0-2a13-4382-8074-1902221b2c10 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:48.031863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab906d6f-da5d-40c3-b864-982f7f295520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:48.032800 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab906d6f-da5d-40c3-b864-982f7f295520 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab906d6f-da5d-40c3-b864-982f7f295520 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.033364 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab906d6f-da5d-40c3-b864-982f7f295520 None admin] PATH_INFO: `/v3/users/22c8ec1706ac451a946494eb8709ee0a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.049914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab906d6f-da5d-40c3-b864-982f7f295520 None admin] RBAC: Authorizing `identity:get_user(user_id=22c8ec1706ac451a946494eb8709ee0a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:48.051498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab906d6f-da5d-40c3-b864-982f7f295520 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:48.053609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 121/243] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:47 2022] GET /identity/v3/users/22c8ec1706ac451a946494eb8709ee0a => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:48.060177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42577a-7680-42eb-9769-0831459a02de None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:48.077927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7974af7-3dd6-4063-a916-9a4a4e67d01f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.078222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7974af7-3dd6-4063-a916-9a4a4e67d01f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.078626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:48.079099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7974af7-3dd6-4063-a916-9a4a4e67d01f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.080527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 122/244] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:48.085876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073ada2-2228-4c8e-a94a-d8b5f0eb3f16 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.086223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073ada2-2228-4c8e-a94a-d8b5f0eb3f16 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.086504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073ada2-2228-4c8e-a94a-d8b5f0eb3f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.107962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c073ada2-2228-4c8e-a94a-d8b5f0eb3f16 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:48.109260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c073ada2-2228-4c8e-a94a-d8b5f0eb3f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:48.199798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:48.206503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 123/245] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:48.220112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42577a-7680-42eb-9769-0831459a02de None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:48.231735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073ada2-2228-4c8e-a94a-d8b5f0eb3f16 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:48.259593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e82c0-a6ea-4f25-8d13-342efccca292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:48.260708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e82c0-a6ea-4f25-8d13-342efccca292 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.261368 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e82c0-a6ea-4f25-8d13-342efccca292 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.261793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e82c0-a6ea-4f25-8d13-342efccca292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.351682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e82c0-a6ea-4f25-8d13-342efccca292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:48.353420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e82c0-a6ea-4f25-8d13-342efccca292 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:48.363623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073ada2-2228-4c8e-a94a-d8b5f0eb3f16 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:48.373481 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:48.412822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c5a5d-edcf-4eff-b829-0dce2dea1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:48.413626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c5a5d-edcf-4eff-b829-0dce2dea1da3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.414352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c5a5d-edcf-4eff-b829-0dce2dea1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.414918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c5a5d-edcf-4eff-b829-0dce2dea1da3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.429537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c5a5d-edcf-4eff-b829-0dce2dea1da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-190c5a5d-edcf-4eff-b829-0dce2dea1da3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:48.440422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:48.441496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:48.442555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:48.445517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 124/246] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:48.460416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c5a5d-edcf-4eff-b829-0dce2dea1da3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:48.491071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 123/247] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:48.497519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e82c0-a6ea-4f25-8d13-342efccca292 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:48.607915 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e82c0-a6ea-4f25-8d13-342efccca292 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:48.622453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:48.624109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:48.647622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb19a6-a62e-4317-9b98-7704ec0a9b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:48.648625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb19a6-a62e-4317-9b98-7704ec0a9b39 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.649069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb19a6-a62e-4317-9b98-7704ec0a9b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.649418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb19a6-a62e-4317-9b98-7704ec0a9b39 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.650736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb19a6-a62e-4317-9b98-7704ec0a9b39 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:48.652435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb19a6-a62e-4317-9b98-7704ec0a9b39 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:48.654689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 124/248] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:48 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:48.663318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb19a6-a62e-4317-9b98-7704ec0a9b39 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:48.761472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c5a5d-edcf-4eff-b829-0dce2dea1da3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:48.765122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb19a6-a62e-4317-9b98-7704ec0a9b39 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:48.802737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225cf17-b810-4bc5-bcc7-07b1428428d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:48.803527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225cf17-b810-4bc5-bcc7-07b1428428d2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.803882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225cf17-b810-4bc5-bcc7-07b1428428d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.804360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225cf17-b810-4bc5-bcc7-07b1428428d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.805409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225cf17-b810-4bc5-bcc7-07b1428428d2 None admin] RBAC: Adding query filter params (role.id=3ff68e8f3bb340bf985c55b19b58c222, scope.project.id=4b61245437cd4d14997cf3df4ea706f0, user.id=22c8ec1706ac451a946494eb8709ee0a) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:48.805791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225cf17-b810-4bc5-bcc7-07b1428428d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:48.808074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225cf17-b810-4bc5-bcc7-07b1428428d2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:48.815690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 125/249] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:48 2022] GET /identity/v3/role_assignments?user.id=22c8ec1706ac451a946494eb8709ee0a&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=3ff68e8f3bb340bf985c55b19b58c222 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:48.849167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdef04c-b538-479e-b6e6-ec41b5b1a017 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.849447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdef04c-b538-479e-b6e6-ec41b5b1a017 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.849726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdef04c-b538-479e-b6e6-ec41b5b1a017 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.850020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441704a-c61c-4ca6-b8a6-fffbe2b505df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:48.850700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441704a-c61c-4ca6-b8a6-fffbe2b505df None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.850975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441704a-c61c-4ca6-b8a6-fffbe2b505df None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.851157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 126/250] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:20:48.851752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441704a-c61c-4ca6-b8a6-fffbe2b505df None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.852511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441704a-c61c-4ca6-b8a6-fffbe2b505df None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:48.852878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441704a-c61c-4ca6-b8a6-fffbe2b505df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:48.854379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441704a-c61c-4ca6-b8a6-fffbe2b505df None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:48.857513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645111d-a9b1-4902-9511-9a909e5b9655 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.857801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645111d-a9b1-4902-9511-9a909e5b9655 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.858095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5645111d-a9b1-4902-9511-9a909e5b9655 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.859711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 127/251] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:48.861258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66830e-418f-40a0-93b4-80459347ee5b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.861697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66830e-418f-40a0-93b4-80459347ee5b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.862412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66830e-418f-40a0-93b4-80459347ee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.864091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 125/252] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:48.869241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fba76f-4b18-4f5e-a37f-5df7bad1d110 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:48.869627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fba76f-4b18-4f5e-a37f-5df7bad1d110 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:48.870009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fba76f-4b18-4f5e-a37f-5df7bad1d110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:48.887857 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c5fba76f-4b18-4f5e-a37f-5df7bad1d110 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:48.889043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c5fba76f-4b18-4f5e-a37f-5df7bad1d110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:48.889660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-cd66830e-418f-40a0-93b4-80459347ee5b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:48.890847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cd66830e-418f-40a0-93b4-80459347ee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:48.943904 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:48.987996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 128/253] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:48.988879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66830e-418f-40a0-93b4-80459347ee5b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:49.015237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 126/254] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:49.028294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fba76f-4b18-4f5e-a37f-5df7bad1d110 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:49.092532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:49.175384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66830e-418f-40a0-93b4-80459347ee5b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:49.178452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fba76f-4b18-4f5e-a37f-5df7bad1d110 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:49.221792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef885ee4-efd7-4c3a-bce9-0f35d6bf8e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:49.223114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef885ee4-efd7-4c3a-bce9-0f35d6bf8e9c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:49.224596 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef885ee4-efd7-4c3a-bce9-0f35d6bf8e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:49.226304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef885ee4-efd7-4c3a-bce9-0f35d6bf8e9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:49.248346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef885ee4-efd7-4c3a-bce9-0f35d6bf8e9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:49.249800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e7a4-7a4d-46ad-a919-b805ebd07dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:49.252083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e7a4-7a4d-46ad-a919-b805ebd07dca None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:49.255714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e7a4-7a4d-46ad-a919-b805ebd07dca None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:49.256187 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67e7a4-7a4d-46ad-a919-b805ebd07dca None admin] PATH_INFO: `/v3/users/d0b19bfb85b74fb0b4d118bb192bfb53` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:49.257596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67e7a4-7a4d-46ad-a919-b805ebd07dca None admin] RBAC: Authorizing `identity:get_user(user_id=d0b19bfb85b74fb0b4d118bb192bfb53)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:49.259280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67e7a4-7a4d-46ad-a919-b805ebd07dca None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:49.264888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 129/255] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:48 2022] GET /identity/v3/users/d0b19bfb85b74fb0b4d118bb192bfb53 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:49.265907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e7a4-7a4d-46ad-a919-b805ebd07dca None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ef885ee4-efd7-4c3a-bce9-0f35d6bf8e9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:49.270671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:49.271560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 127/256] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:20:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:49.272329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef885ee4-efd7-4c3a-bce9-0f35d6bf8e9c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:49.323104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:49.387892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67e7a4-7a4d-46ad-a919-b805ebd07dca None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:49.424923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895ff5a-f36b-432e-aeec-e2555d31fd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:49.426228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895ff5a-f36b-432e-aeec-e2555d31fd3f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:49.426722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895ff5a-f36b-432e-aeec-e2555d31fd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:49.427186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895ff5a-f36b-432e-aeec-e2555d31fd3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:49.438850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895ff5a-f36b-432e-aeec-e2555d31fd3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2895ff5a-f36b-432e-aeec-e2555d31fd3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:49.447580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:49.448790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:20:49.449566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:49.453228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 130/257] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:20:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:49.453228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895ff5a-f36b-432e-aeec-e2555d31fd3f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:49.565562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef885ee4-efd7-4c3a-bce9-0f35d6bf8e9c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:49.585243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:49.587222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895ff5a-f36b-432e-aeec-e2555d31fd3f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:49.589111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:49.622745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b48809-d125-42f8-a532-aad6512ecbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:49.623454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48809-d125-42f8-a532-aad6512ecbbc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:49.623735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48809-d125-42f8-a532-aad6512ecbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:49.624010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48809-d125-42f8-a532-aad6512ecbbc None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:49.624759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b48809-d125-42f8-a532-aad6512ecbbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:49.625069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b48809-d125-42f8-a532-aad6512ecbbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:49.626458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b48809-d125-42f8-a532-aad6512ecbbc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:49.634801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 131/258] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:20:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:49.635551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b48809-d125-42f8-a532-aad6512ecbbc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:49.655898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c27d1-0d3e-4ca6-97ac-793635a55854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:49.657116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c27d1-0d3e-4ca6-97ac-793635a55854 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:49.657609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c27d1-0d3e-4ca6-97ac-793635a55854 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:49.658119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066c27d1-0d3e-4ca6-97ac-793635a55854 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:49.660875 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c27d1-0d3e-4ca6-97ac-793635a55854 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:49.662600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066c27d1-0d3e-4ca6-97ac-793635a55854 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:49.668061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 128/259] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:49 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:49.669116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c27d1-0d3e-4ca6-97ac-793635a55854 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:49.757239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b48809-d125-42f8-a532-aad6512ecbbc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:49.806531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066c27d1-0d3e-4ca6-97ac-793635a55854 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:49.831709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188e05-47e9-4a4f-a23f-a3ae5630fb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:49.832313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188e05-47e9-4a4f-a23f-a3ae5630fb68 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:49.832597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188e05-47e9-4a4f-a23f-a3ae5630fb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:49.832871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12188e05-47e9-4a4f-a23f-a3ae5630fb68 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:49.833632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12188e05-47e9-4a4f-a23f-a3ae5630fb68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:49.833940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12188e05-47e9-4a4f-a23f-a3ae5630fb68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:49.835365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12188e05-47e9-4a4f-a23f-a3ae5630fb68 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:49.849445 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 132/260] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:49.850268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188e05-47e9-4a4f-a23f-a3ae5630fb68 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:49.860340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47001bf8-1dbb-4946-9aad-82325d945952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:49.861233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47001bf8-1dbb-4946-9aad-82325d945952 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:49.861515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47001bf8-1dbb-4946-9aad-82325d945952 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:49.861791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47001bf8-1dbb-4946-9aad-82325d945952 None admin] PATH_INFO: `/v3/users/11d0038b8824424b814f6ab7ab0f3d0c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:49.863711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47001bf8-1dbb-4946-9aad-82325d945952 None admin] RBAC: Authorizing `identity:get_user(user_id=11d0038b8824424b814f6ab7ab0f3d0c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:49.865176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47001bf8-1dbb-4946-9aad-82325d945952 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:49.866840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 129/261] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:49 2022] GET /identity/v3/users/11d0038b8824424b814f6ab7ab0f3d0c => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:49.867623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47001bf8-1dbb-4946-9aad-82325d945952 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:49.898692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:49.969400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:50.016871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12188e05-47e9-4a4f-a23f-a3ae5630fb68 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:50.053347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47001bf8-1dbb-4946-9aad-82325d945952 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:50.077450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6b6db-3eda-4c3c-9701-b0b74281c41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:50.078348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6b6db-3eda-4c3c-9701-b0b74281c41c None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:50.078626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6b6db-3eda-4c3c-9701-b0b74281c41c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:50.078898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6b6db-3eda-4c3c-9701-b0b74281c41c None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0/users/d0b19bfb85b74fb0b4d118bb192bfb53/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:50.081893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6b6db-3eda-4c3c-9701-b0b74281c41c None admin] RBAC: Authorizing `identity:create_grant(project_id=4b61245437cd4d14997cf3df4ea706f0, user_id=d0b19bfb85b74fb0b4d118bb192bfb53, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:50.084482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6b6db-3eda-4c3c-9701-b0b74281c41c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:50.102870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2043f-b21e-4297-83e5-d03c2f7a69d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:50.104057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2043f-b21e-4297-83e5-d03c2f7a69d9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:50.104515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2043f-b21e-4297-83e5-d03c2f7a69d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:50.104854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2043f-b21e-4297-83e5-d03c2f7a69d9 None admin] PATH_INFO: `/v3/users/11d0038b8824424b814f6ab7ab0f3d0c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:50.106297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2043f-b21e-4297-83e5-d03c2f7a69d9 None admin] RBAC: Authorizing `identity:get_user(user_id=11d0038b8824424b814f6ab7ab0f3d0c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:50.107993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2043f-b21e-4297-83e5-d03c2f7a69d9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:50.110137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 130/262] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:49 2022] GET /identity/v3/users/11d0038b8824424b814f6ab7ab0f3d0c => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:50.110914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2043f-b21e-4297-83e5-d03c2f7a69d9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:50.121559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 133/263] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:20:49 2022] PUT /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0/users/d0b19bfb85b74fb0b4d118bb192bfb53/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:20:50.122395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6b6db-3eda-4c3c-9701-b0b74281c41c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:50.268910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2043f-b21e-4297-83e5-d03c2f7a69d9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:50.299214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:50.335484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c6f5c-de87-4176-8114-f54f8a018482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:50.336489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c6f5c-de87-4176-8114-f54f8a018482 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:50.336970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c6f5c-de87-4176-8114-f54f8a018482 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:50.337462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c6f5c-de87-4176-8114-f54f8a018482 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:50.356391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:50.383418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6b6db-3eda-4c3c-9701-b0b74281c41c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:50.442342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d184238-e330-461f-a978-1e228db9fca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:50.443377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d184238-e330-461f-a978-1e228db9fca7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:50.443861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d184238-e330-461f-a978-1e228db9fca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:50.445497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d184238-e330-461f-a978-1e228db9fca7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:50.483923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c6f5c-de87-4176-8114-f54f8a018482 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:50.485780 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c6f5c-de87-4176-8114-f54f8a018482 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:50.603577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d184238-e330-461f-a978-1e228db9fca7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:50.606029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d184238-e330-461f-a978-1e228db9fca7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:50.706664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 131/264] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:50.710424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd82302-dae8-4241-8bcc-a8e81853fdec None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:50.710424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd82302-dae8-4241-8bcc-a8e81853fdec None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:50.710424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd82302-dae8-4241-8bcc-a8e81853fdec None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:50.710424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 132/265] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:50.720745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbeed60-51a2-4cc0-a0cb-bcf3a7641c56 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:50.721025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbeed60-51a2-4cc0-a0cb-bcf3a7641c56 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:50.721300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbeed60-51a2-4cc0-a0cb-bcf3a7641c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:50.740013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:50.750577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dbbeed60-51a2-4cc0-a0cb-bcf3a7641c56 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:50.751840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dbbeed60-51a2-4cc0-a0cb-bcf3a7641c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:50.759495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 134/266] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:50.761429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d184238-e330-461f-a978-1e228db9fca7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:50.775685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:50.884384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d184238-e330-461f-a978-1e228db9fca7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:50.904026 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 133/267] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:50.904971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbeed60-51a2-4cc0-a0cb-bcf3a7641c56 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:50.930454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55b916-a5fc-4230-bd45-72c85d499071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:50.931346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55b916-a5fc-4230-bd45-72c85d499071 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:50.931925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55b916-a5fc-4230-bd45-72c85d499071 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:50.932322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55b916-a5fc-4230-bd45-72c85d499071 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:50.933770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55b916-a5fc-4230-bd45-72c85d499071 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:50.935243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55b916-a5fc-4230-bd45-72c85d499071 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:50.938006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 135/268] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:50 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:50.939261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca13d9-e169-4457-b776-6da852f18c75 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:50.939568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca13d9-e169-4457-b776-6da852f18c75 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:50.939838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca13d9-e169-4457-b776-6da852f18c75 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:50.941069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 136/269] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:50.941746 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca13d9-e169-4457-b776-6da852f18c75 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:51.018504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:51.026943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbeed60-51a2-4cc0-a0cb-bcf3a7641c56 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:51.069560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca13d9-e169-4457-b776-6da852f18c75 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:51.073116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d58c3-1a9a-4571-865e-87ef124bb806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:51.074316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d58c3-1a9a-4571-865e-87ef124bb806 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.074785 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d58c3-1a9a-4571-865e-87ef124bb806 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.075242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d58c3-1a9a-4571-865e-87ef124bb806 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.076780 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d58c3-1a9a-4571-865e-87ef124bb806 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:51.079010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d58c3-1a9a-4571-865e-87ef124bb806 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:51.087057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 134/270] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:50 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:51.088779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d58c3-1a9a-4571-865e-87ef124bb806 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:51.109407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:51.112086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c6e1b-e92e-477a-be75-466e26146d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:51.113361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c6e1b-e92e-477a-be75-466e26146d9d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.113657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c6e1b-e92e-477a-be75-466e26146d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.113932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c6e1b-e92e-477a-be75-466e26146d9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.130104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c6e1b-e92e-477a-be75-466e26146d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-826c6e1b-e92e-477a-be75-466e26146d9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:51.140671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:51.141551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:51.142323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 137/271] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:51.143482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1faabe-d4a9-4a5c-80d0-d470e8ccadb2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.143482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1faabe-d4a9-4a5c-80d0-d470e8ccadb2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.143642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1faabe-d4a9-4a5c-80d0-d470e8ccadb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.166839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6c1faabe-d4a9-4a5c-80d0-d470e8ccadb2 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:51.168417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6c1faabe-d4a9-4a5c-80d0-d470e8ccadb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:51.216196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d58c3-1a9a-4571-865e-87ef124bb806 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:51.238722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 138/272] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:51.239936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1faabe-d4a9-4a5c-80d0-d470e8ccadb2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:51.264758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230ffe2-d845-4789-8333-cd5f3b3bf031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:51.265406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230ffe2-d845-4789-8333-cd5f3b3bf031 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.265684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230ffe2-d845-4789-8333-cd5f3b3bf031 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.265990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230ffe2-d845-4789-8333-cd5f3b3bf031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.266835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230ffe2-d845-4789-8333-cd5f3b3bf031 None admin] RBAC: Adding query filter params (role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=4b61245437cd4d14997cf3df4ea706f0, user.id=11d0038b8824424b814f6ab7ab0f3d0c) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:51.267144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230ffe2-d845-4789-8333-cd5f3b3bf031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:51.268631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230ffe2-d845-4789-8333-cd5f3b3bf031 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:51.279779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 135/273] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:51 2022] GET /identity/v3/role_assignments?user.id=11d0038b8824424b814f6ab7ab0f3d0c&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=62ac4326d6d742c5b6684746953362b1 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:51.287435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230ffe2-d845-4789-8333-cd5f3b3bf031 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:51.303510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:51.399884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:51.420381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230ffe2-d845-4789-8333-cd5f3b3bf031 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:51.462728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da4517-e696-43bb-bc47-97823143494e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:51.464014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da4517-e696-43bb-bc47-97823143494e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.464443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da4517-e696-43bb-bc47-97823143494e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.464898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da4517-e696-43bb-bc47-97823143494e None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.465653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da4517-e696-43bb-bc47-97823143494e None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:51.465959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da4517-e696-43bb-bc47-97823143494e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:51.467365 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da4517-e696-43bb-bc47-97823143494e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:51.478238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 136/274] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:51.483223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da4517-e696-43bb-bc47-97823143494e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:51.494086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1faabe-d4a9-4a5c-80d0-d470e8ccadb2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:51.536011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd231-34af-4cc9-87fe-a1c9409736a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:51.537035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd231-34af-4cc9-87fe-a1c9409736a5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.537507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd231-34af-4cc9-87fe-a1c9409736a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.537994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd231-34af-4cc9-87fe-a1c9409736a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.539018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd231-34af-4cc9-87fe-a1c9409736a5 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=4b61245437cd4d14997cf3df4ea706f0, user.id=11d0038b8824424b814f6ab7ab0f3d0c) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:51.539541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd231-34af-4cc9-87fe-a1c9409736a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:51.542229 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341fd231-34af-4cc9-87fe-a1c9409736a5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:51.549068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 139/275] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:51 2022] GET /identity/v3/role_assignments?user.id=11d0038b8824424b814f6ab7ab0f3d0c&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:51.550147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd231-34af-4cc9-87fe-a1c9409736a5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:51.617314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da4517-e696-43bb-bc47-97823143494e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:51.670954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f95994-2a36-4aea-87ec-253f1e17ce49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:51.671729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f95994-2a36-4aea-87ec-253f1e17ce49 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.672054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f95994-2a36-4aea-87ec-253f1e17ce49 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.672305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f95994-2a36-4aea-87ec-253f1e17ce49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.673498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f95994-2a36-4aea-87ec-253f1e17ce49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:51.674974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f95994-2a36-4aea-87ec-253f1e17ce49 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:51.677064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 137/276] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:51.678348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e203465-7c9c-4799-95e9-4eff07bf2448 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.678625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e203465-7c9c-4799-95e9-4eff07bf2448 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.678901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e203465-7c9c-4799-95e9-4eff07bf2448 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.685768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 138/277] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:51.686600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e203465-7c9c-4799-95e9-4eff07bf2448 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:51.699068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:51.710912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341fd231-34af-4cc9-87fe-a1c9409736a5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:51.721941 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:51.752916 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8701b6-f169-467b-918c-24e540f085cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:51.753804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8701b6-f169-467b-918c-24e540f085cd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.754087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8701b6-f169-467b-918c-24e540f085cd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.754364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8701b6-f169-467b-918c-24e540f085cd None admin] PATH_INFO: `/v3/users/c08d14c4ed964998bc44513160b9374f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.772006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8701b6-f169-467b-918c-24e540f085cd None admin] RBAC: Authorizing `identity:get_user(user_id=c08d14c4ed964998bc44513160b9374f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:51.773492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8701b6-f169-467b-918c-24e540f085cd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:51.775227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 140/278] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:51 2022] GET /identity/v3/users/c08d14c4ed964998bc44513160b9374f => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:51.776666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9310bd7-d762-4602-b124-b0035f272a49 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.776953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9310bd7-d762-4602-b124-b0035f272a49 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.777228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9310bd7-d762-4602-b124-b0035f272a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.794173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c9310bd7-d762-4602-b124-b0035f272a49 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:51.795395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c9310bd7-d762-4602-b124-b0035f272a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:51.824520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e203465-7c9c-4799-95e9-4eff07bf2448 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:51.874657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8145117-c4f1-4275-8432-6ae90ef8f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:51.875553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8145117-c4f1-4275-8432-6ae90ef8f8b0 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:51.875936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8145117-c4f1-4275-8432-6ae90ef8f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:51.876218 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8145117-c4f1-4275-8432-6ae90ef8f8b0 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:51.876958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8145117-c4f1-4275-8432-6ae90ef8f8b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:51.878404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8145117-c4f1-4275-8432-6ae90ef8f8b0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:51.882249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 141/279] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:51.882933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9310bd7-d762-4602-b124-b0035f272a49 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:51.906576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 139/280] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:20:51 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:51.907794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8145117-c4f1-4275-8432-6ae90ef8f8b0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:51.978866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:51.993913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:51.998024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9310bd7-d762-4602-b124-b0035f272a49 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:52.026855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8145117-c4f1-4275-8432-6ae90ef8f8b0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:52.036223 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5912-9ed9-4e20-9c72-f339de687c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:52.036774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5912-9ed9-4e20-9c72-f339de687c30 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:52.037067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5912-9ed9-4e20-9c72-f339de687c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:52.037344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5912-9ed9-4e20-9c72-f339de687c30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:52.038206 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f5912-9ed9-4e20-9c72-f339de687c30 None admin] RBAC: Adding query filter params (role.id=6aab4b825fea4f4292adc656ae17a709, user.id=c08d14c4ed964998bc44513160b9374f, scope.system=all) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:52.038520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f5912-9ed9-4e20-9c72-f339de687c30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:52.040305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f5912-9ed9-4e20-9c72-f339de687c30 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:52.046676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 142/281] 10.222.0.42 () {56 vars in 1308 bytes} [Sat Jul 2 03:20:51 2022] GET /identity/v3/role_assignments?user.id=c08d14c4ed964998bc44513160b9374f&scope.system=all&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:52.069939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ade243-aa51-49e5-8f8f-77adaf079bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:52.071034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ade243-aa51-49e5-8f8f-77adaf079bbd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:52.071499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ade243-aa51-49e5-8f8f-77adaf079bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:52.071909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ade243-aa51-49e5-8f8f-77adaf079bbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:52.085190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ade243-aa51-49e5-8f8f-77adaf079bbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-35ade243-aa51-49e5-8f8f-77adaf079bbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:52.093512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:52.094604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:52.095580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:52.097914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 140/282] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:52.104196 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5912-9ed9-4e20-9c72-f339de687c30 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:52.222248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5912-9ed9-4e20-9c72-f339de687c30 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:52.259845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada03de-6d8c-42ed-a39e-4e43da26059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:52.260799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada03de-6d8c-42ed-a39e-4e43da26059a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:52.261217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada03de-6d8c-42ed-a39e-4e43da26059a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:52.261624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada03de-6d8c-42ed-a39e-4e43da26059a None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:52.262501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada03de-6d8c-42ed-a39e-4e43da26059a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:52.262956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada03de-6d8c-42ed-a39e-4e43da26059a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:52.264518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada03de-6d8c-42ed-a39e-4e43da26059a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:52.274689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 143/283] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:52.280549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ade243-aa51-49e5-8f8f-77adaf079bbd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:52.388870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ade243-aa51-49e5-8f8f-77adaf079bbd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:52.412415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:52.424908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb6ef4-99d8-4adc-acfe-889388d83bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:52.426073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb6ef4-99d8-4adc-acfe-889388d83bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:52.426486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb6ef4-99d8-4adc-acfe-889388d83bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:52.426889 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb6ef4-99d8-4adc-acfe-889388d83bc3 None admin] PATH_INFO: `/v3/users/22c8ec1706ac451a946494eb8709ee0a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:52.428241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb6ef4-99d8-4adc-acfe-889388d83bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=22c8ec1706ac451a946494eb8709ee0a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:52.429901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb6ef4-99d8-4adc-acfe-889388d83bc3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:52.432487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 141/284] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:52 2022] GET /identity/v3/users/22c8ec1706ac451a946494eb8709ee0a => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:20:52.437861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada03de-6d8c-42ed-a39e-4e43da26059a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:52.497018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:52.596288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada03de-6d8c-42ed-a39e-4e43da26059a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:52.603582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ead56d-0f8d-4db2-bda3-904ebabef065 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:52.604261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ead56d-0f8d-4db2-bda3-904ebabef065 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:52.604725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ead56d-0f8d-4db2-bda3-904ebabef065 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:52.606298 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 142/285] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:20:52.611510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7df9d-4c25-4c9f-a997-3ae942545348 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:52.611801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7df9d-4c25-4c9f-a997-3ae942545348 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:52.612084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7df9d-4c25-4c9f-a997-3ae942545348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:52.636761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0556a-126c-422d-b5a0-18b7c7c23412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:52.637653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0556a-126c-422d-b5a0-18b7c7c23412 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:52.637933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0556a-126c-422d-b5a0-18b7c7c23412 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:52.638209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0556a-126c-422d-b5a0-18b7c7c23412 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:52.639447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0556a-126c-422d-b5a0-18b7c7c23412 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:52.640907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0556a-126c-422d-b5a0-18b7c7c23412 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:52.642634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 144/286] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:52 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:52.651945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0556a-126c-422d-b5a0-18b7c7c23412 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:52.653620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-59c7df9d-4c25-4c9f-a997-3ae942545348 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:52.654833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-59c7df9d-4c25-4c9f-a997-3ae942545348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:52.790918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 143/287] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:52.801352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:52.811079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0556a-126c-422d-b5a0-18b7c7c23412 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:52.826215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7df9d-4c25-4c9f-a997-3ae942545348 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:52.874339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060a873-3785-41a0-847b-ab26bd6760a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:52.875164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060a873-3785-41a0-847b-ab26bd6760a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:52.875474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060a873-3785-41a0-847b-ab26bd6760a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:52.875752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060a873-3785-41a0-847b-ab26bd6760a9 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0/users/22c8ec1706ac451a946494eb8709ee0a/roles/3ff68e8f3bb340bf985c55b19b58c222` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:52.882682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060a873-3785-41a0-847b-ab26bd6760a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b61245437cd4d14997cf3df4ea706f0, user_id=22c8ec1706ac451a946494eb8709ee0a, role_id=3ff68e8f3bb340bf985c55b19b58c222)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:52.887789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060a873-3785-41a0-847b-ab26bd6760a9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:52.923452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 145/288] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:20:52 2022] PUT /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0/users/22c8ec1706ac451a946494eb8709ee0a/roles/3ff68e8f3bb340bf985c55b19b58c222 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:20:52.976978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7df9d-4c25-4c9f-a997-3ae942545348 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:53.000989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:53.022712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25808f7-83a0-4703-895a-758a435ead70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:53.023655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25808f7-83a0-4703-895a-758a435ead70 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:53.023945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25808f7-83a0-4703-895a-758a435ead70 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:53.024222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25808f7-83a0-4703-895a-758a435ead70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:53.041488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25808f7-83a0-4703-895a-758a435ead70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d25808f7-83a0-4703-895a-758a435ead70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:53.053908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:53.054836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:53.055668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 144/289] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:53.061821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060a873-3785-41a0-847b-ab26bd6760a9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:53.178391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060a873-3785-41a0-847b-ab26bd6760a9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:53.227143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310d797-f826-47b0-bbf1-4874d704531d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:53.228256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310d797-f826-47b0-bbf1-4874d704531d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:53.228824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310d797-f826-47b0-bbf1-4874d704531d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:53.229343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310d797-f826-47b0-bbf1-4874d704531d None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:53.230297 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310d797-f826-47b0-bbf1-4874d704531d None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:53.230790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310d797-f826-47b0-bbf1-4874d704531d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:53.232390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310d797-f826-47b0-bbf1-4874d704531d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:53.241390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 146/290] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:53.246591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25808f7-83a0-4703-895a-758a435ead70 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:53.370535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25808f7-83a0-4703-895a-758a435ead70 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:53.421788 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e92b-c35d-468b-89e5-1a58528f890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:53.423040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e92b-c35d-468b-89e5-1a58528f890c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:53.423537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e92b-c35d-468b-89e5-1a58528f890c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:53.424001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10e92b-c35d-468b-89e5-1a58528f890c None admin] PATH_INFO: `/v3/users/d0b19bfb85b74fb0b4d118bb192bfb53` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:53.425414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10e92b-c35d-468b-89e5-1a58528f890c None admin] RBAC: Authorizing `identity:get_user(user_id=d0b19bfb85b74fb0b4d118bb192bfb53)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:53.427048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c10e92b-c35d-468b-89e5-1a58528f890c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:53.428962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 145/291] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:53 2022] GET /identity/v3/users/d0b19bfb85b74fb0b4d118bb192bfb53 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:53.434776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310d797-f826-47b0-bbf1-4874d704531d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:53.451343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:53.588105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310d797-f826-47b0-bbf1-4874d704531d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:53.621462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81942c7b-52de-4aa0-bc5c-e05e06d06d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:53.622111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942c7b-52de-4aa0-bc5c-e05e06d06d53 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:53.622399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942c7b-52de-4aa0-bc5c-e05e06d06d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:53.622676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942c7b-52de-4aa0-bc5c-e05e06d06d53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:53.710666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81942c7b-52de-4aa0-bc5c-e05e06d06d53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:53.712091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81942c7b-52de-4aa0-bc5c-e05e06d06d53 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:53.736718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:53.819989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 147/292] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:53.825607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e92b-c35d-468b-89e5-1a58528f890c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:53.903088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:53.983213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10e92b-c35d-468b-89e5-1a58528f890c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:54.030890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2effbc-e3f4-444f-a661-af18043d0215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:54.031788 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2effbc-e3f4-444f-a661-af18043d0215 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.032063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2effbc-e3f4-444f-a661-af18043d0215 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.032334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2effbc-e3f4-444f-a661-af18043d0215 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.035980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2effbc-e3f4-444f-a661-af18043d0215 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:54.037470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2effbc-e3f4-444f-a661-af18043d0215 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:54.039523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 146/293] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:53 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:54.045446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81942c7b-52de-4aa0-bc5c-e05e06d06d53 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:54.162805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81942c7b-52de-4aa0-bc5c-e05e06d06d53 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:54.181129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:54.212046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e68a3-7ecc-4075-94ec-2af488cdb83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:54.212975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e68a3-7ecc-4075-94ec-2af488cdb83f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.213496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e68a3-7ecc-4075-94ec-2af488cdb83f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.213902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e68a3-7ecc-4075-94ec-2af488cdb83f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.215201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e68a3-7ecc-4075-94ec-2af488cdb83f None admin] RBAC: Adding query filter params (role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=4b61245437cd4d14997cf3df4ea706f0, user.id=d0b19bfb85b74fb0b4d118bb192bfb53) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:54.215690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e68a3-7ecc-4075-94ec-2af488cdb83f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:54.217913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e68a3-7ecc-4075-94ec-2af488cdb83f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:54.227577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 148/294] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:54 2022] GET /identity/v3/role_assignments?user.id=d0b19bfb85b74fb0b4d118bb192bfb53&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:54.473582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531b898-65bb-4cd8-b0d0-566b6c68d527 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.474275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531b898-65bb-4cd8-b0d0-566b6c68d527 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.474275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531b898-65bb-4cd8-b0d0-566b6c68d527 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.475714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 147/295] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:54.481352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa708b7-67e4-4ac2-b13d-59da0136b9ca None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.481644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa708b7-67e4-4ac2-b13d-59da0136b9ca None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.481894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa708b7-67e4-4ac2-b13d-59da0136b9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.504382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9aa708b7-67e4-4ac2-b13d-59da0136b9ca None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:54.505600 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9aa708b7-67e4-4ac2-b13d-59da0136b9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:54.520604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60d99e-b982-43f5-a57d-a4584f532824 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.520893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60d99e-b982-43f5-a57d-a4584f532824 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.521163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60d99e-b982-43f5-a57d-a4584f532824 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.522805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 148/296] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:54.530133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad463db-4ee8-4b59-bafe-37c9e8f83b8a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.530423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad463db-4ee8-4b59-bafe-37c9e8f83b8a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.530707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad463db-4ee8-4b59-bafe-37c9e8f83b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.553094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0ad463db-4ee8-4b59-bafe-37c9e8f83b8a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:54.554406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0ad463db-4ee8-4b59-bafe-37c9e8f83b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:54.585675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 149/297] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:54.615794 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa708b7-67e4-4ac2-b13d-59da0136b9ca None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:54.643491 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 149/298] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:54.661684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad463db-4ee8-4b59-bafe-37c9e8f83b8a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:54.714310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:54.719860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa708b7-67e4-4ac2-b13d-59da0136b9ca None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:54.727612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:54.755839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc64b85-fe1d-420e-bd35-d93100dbf299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:54.756635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc64b85-fe1d-420e-bd35-d93100dbf299 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.756913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc64b85-fe1d-420e-bd35-d93100dbf299 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.757183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc64b85-fe1d-420e-bd35-d93100dbf299 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.768328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc64b85-fe1d-420e-bd35-d93100dbf299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:54.774443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad463db-4ee8-4b59-bafe-37c9e8f83b8a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bbc64b85-fe1d-420e-bd35-d93100dbf299 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:54.776345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:54.778387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:54.779741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 150/299] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:20:54.788580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc64b85-fe1d-420e-bd35-d93100dbf299 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:54.824004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e78b7b-f32a-4c4c-854e-dc60b5692975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:54.824828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e78b7b-f32a-4c4c-854e-dc60b5692975 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.825113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e78b7b-f32a-4c4c-854e-dc60b5692975 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.825408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e78b7b-f32a-4c4c-854e-dc60b5692975 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.841867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e78b7b-f32a-4c4c-854e-dc60b5692975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b6e78b7b-f32a-4c4c-854e-dc60b5692975 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:54.851507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:54.852397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:54.853182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:20:54.855377 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 150/300] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:54.859025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e78b7b-f32a-4c4c-854e-dc60b5692975 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:54.918535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc64b85-fe1d-420e-bd35-d93100dbf299 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:54.966208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a081b8-7ba8-4332-ad9e-d78c655aa3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:54.966947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e78b7b-f32a-4c4c-854e-dc60b5692975 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:54.967428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a081b8-7ba8-4332-ad9e-d78c655aa3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.967881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a081b8-7ba8-4332-ad9e-d78c655aa3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.968363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a081b8-7ba8-4332-ad9e-d78c655aa3d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.970374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a081b8-7ba8-4332-ad9e-d78c655aa3d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:54.970866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a081b8-7ba8-4332-ad9e-d78c655aa3d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:54.972508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a081b8-7ba8-4332-ad9e-d78c655aa3d9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:54.974869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:54.982531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 151/301] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:54.985501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02b0b-ca07-41e1-9c84-c7e423578ee6 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:54.985789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02b0b-ca07-41e1-9c84-c7e423578ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:54.986070 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c02b0b-ca07-41e1-9c84-c7e423578ee6 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:54.987530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 152/302] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:54.988226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c02b0b-ca07-41e1-9c84-c7e423578ee6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:55.007395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df04dec-920a-4e8e-887d-df82229c3b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:55.008705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df04dec-920a-4e8e-887d-df82229c3b71 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.009190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df04dec-920a-4e8e-887d-df82229c3b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.009684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df04dec-920a-4e8e-887d-df82229c3b71 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.010828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df04dec-920a-4e8e-887d-df82229c3b71 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:55.011376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df04dec-920a-4e8e-887d-df82229c3b71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:55.012633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df04dec-920a-4e8e-887d-df82229c3b71 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:55.017773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:55.021896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 151/303] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:55.024633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bef015-8826-4e8d-ab85-19d25c6006f8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.025069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bef015-8826-4e8d-ab85-19d25c6006f8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.025347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bef015-8826-4e8d-ab85-19d25c6006f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.044391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b7bef015-8826-4e8d-ab85-19d25c6006f8 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:55.045773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b7bef015-8826-4e8d-ab85-19d25c6006f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:55.097842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c02b0b-ca07-41e1-9c84-c7e423578ee6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:55.115256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 152/304] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:55.116919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bef015-8826-4e8d-ab85-19d25c6006f8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:55.140737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399f800-7404-41cb-8d7b-e2194dac046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:55.141793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399f800-7404-41cb-8d7b-e2194dac046d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.142079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399f800-7404-41cb-8d7b-e2194dac046d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.142355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399f800-7404-41cb-8d7b-e2194dac046d None admin] PATH_INFO: `/v3/users/3748b0e8c8b24ac6accc84aa649df6ff` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.156700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399f800-7404-41cb-8d7b-e2194dac046d None admin] RBAC: Authorizing `identity:get_user(user_id=3748b0e8c8b24ac6accc84aa649df6ff)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:55.158190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399f800-7404-41cb-8d7b-e2194dac046d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:55.159952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 153/305] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:54 2022] GET /identity/v3/users/3748b0e8c8b24ac6accc84aa649df6ff => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:55.165330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399f800-7404-41cb-8d7b-e2194dac046d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:55.234637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:55.238791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bef015-8826-4e8d-ab85-19d25c6006f8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:55.256815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:55.275224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399f800-7404-41cb-8d7b-e2194dac046d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:55.279251 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42cb63-0f95-4755-9497-a1b36482175d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:55.279650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42cb63-0f95-4755-9497-a1b36482175d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.279934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42cb63-0f95-4755-9497-a1b36482175d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.280214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42cb63-0f95-4755-9497-a1b36482175d None admin] PATH_INFO: `/v3/users/c08d14c4ed964998bc44513160b9374f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.281435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42cb63-0f95-4755-9497-a1b36482175d None admin] RBAC: Authorizing `identity:get_user(user_id=c08d14c4ed964998bc44513160b9374f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:55.282894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42cb63-0f95-4755-9497-a1b36482175d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:55.284956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 153/306] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:55 2022] GET /identity/v3/users/c08d14c4ed964998bc44513160b9374f => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:55.285477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42cb63-0f95-4755-9497-a1b36482175d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:55.320144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145ab0d-e65e-495a-a56f-8b067d14dd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:55.320947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145ab0d-e65e-495a-a56f-8b067d14dd6e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.321232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145ab0d-e65e-495a-a56f-8b067d14dd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.321509 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145ab0d-e65e-495a-a56f-8b067d14dd6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.323140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145ab0d-e65e-495a-a56f-8b067d14dd6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:55.324757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145ab0d-e65e-495a-a56f-8b067d14dd6e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:55.326401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 154/307] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:55.327195 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145ab0d-e65e-495a-a56f-8b067d14dd6e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:55.390772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42cb63-0f95-4755-9497-a1b36482175d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:55.425859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145ab0d-e65e-495a-a56f-8b067d14dd6e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:55.426072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8e900-7abe-43ce-8153-8b09f54fab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:55.426733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8e900-7abe-43ce-8153-8b09f54fab4e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.427018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8e900-7abe-43ce-8153-8b09f54fab4e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.427322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8e900-7abe-43ce-8153-8b09f54fab4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.428199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8e900-7abe-43ce-8153-8b09f54fab4e None admin] RBAC: Adding query filter params (role.id=3ff68e8f3bb340bf985c55b19b58c222, user.id=3748b0e8c8b24ac6accc84aa649df6ff, scope.system=all) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:55.428502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8e900-7abe-43ce-8153-8b09f54fab4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:55.429934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8e900-7abe-43ce-8153-8b09f54fab4e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:55.435472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 154/308] 10.222.0.42 () {56 vars in 1308 bytes} [Sat Jul 2 03:20:55 2022] GET /identity/v3/role_assignments?user.id=3748b0e8c8b24ac6accc84aa649df6ff&scope.system=all&role.id=3ff68e8f3bb340bf985c55b19b58c222 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:55.436829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15173834-c7e9-49c9-9bdf-e169ccac0ad4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.437119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15173834-c7e9-49c9-9bdf-e169ccac0ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.437396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15173834-c7e9-49c9-9bdf-e169ccac0ad4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.438671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 155/309] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:55.439414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15173834-c7e9-49c9-9bdf-e169ccac0ad4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:55.474170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:55.475017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cfc4f-4737-4847-bfe4-8dec4ff8e88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:55.475961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cfc4f-4737-4847-bfe4-8dec4ff8e88e None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.476420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cfc4f-4737-4847-bfe4-8dec4ff8e88e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.476630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cfc4f-4737-4847-bfe4-8dec4ff8e88e None admin] PATH_INFO: `/v3/system/users/c08d14c4ed964998bc44513160b9374f/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.478275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cfc4f-4737-4847-bfe4-8dec4ff8e88e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c08d14c4ed964998bc44513160b9374f, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:55.480017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cfc4f-4737-4847-bfe4-8dec4ff8e88e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:55.491166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 155/310] 10.222.0.42 () {58 vars in 1334 bytes} [Sat Jul 2 03:20:55 2022] PUT /identity/v3/system/users/c08d14c4ed964998bc44513160b9374f/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:20:55.492529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c998724-ba03-4c35-9c53-758897e0218b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.492820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c998724-ba03-4c35-9c53-758897e0218b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.493101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c998724-ba03-4c35-9c53-758897e0218b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.522306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0c998724-ba03-4c35-9c53-758897e0218b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:55.524016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0c998724-ba03-4c35-9c53-758897e0218b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:55.541404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:55.586976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15173834-c7e9-49c9-9bdf-e169ccac0ad4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:55.621218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 156/311] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:55.635730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f54d-9736-4c63-bb1b-4d36f9c2a056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:55.636852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f54d-9736-4c63-bb1b-4d36f9c2a056 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.637325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f54d-9736-4c63-bb1b-4d36f9c2a056 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.637846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63f54d-9736-4c63-bb1b-4d36f9c2a056 None admin] PATH_INFO: `/v3/groups` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.638779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63f54d-9736-4c63-bb1b-4d36f9c2a056 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:55.640493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63f54d-9736-4c63-bb1b-4d36f9c2a056 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:55.645100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c998724-ba03-4c35-9c53-758897e0218b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:55.664353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 156/312] 10.222.0.42 () {60 vars in 1134 bytes} [Sat Jul 2 03:20:55 2022] POST /identity/v3/groups => generated 229 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:55.770976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c998724-ba03-4c35-9c53-758897e0218b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:55.814905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3b0d5-f5d7-4320-b2d6-bd1245110805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:55.815746 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3b0d5-f5d7-4320-b2d6-bd1245110805 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.816029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3b0d5-f5d7-4320-b2d6-bd1245110805 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.816302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3b0d5-f5d7-4320-b2d6-bd1245110805 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.828115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3b0d5-f5d7-4320-b2d6-bd1245110805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bad3b0d5-f5d7-4320-b2d6-bd1245110805 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:55.836071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:55.836917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:55.837699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 157/313] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:55.843568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f54d-9736-4c63-bb1b-4d36f9c2a056 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:55.943459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:55.952928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63f54d-9736-4c63-bb1b-4d36f9c2a056 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:55.987968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda9394-532d-4bc6-8b03-5cd5682aca0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:55.988647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda9394-532d-4bc6-8b03-5cd5682aca0f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:55.988924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda9394-532d-4bc6-8b03-5cd5682aca0f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:55.989195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda9394-532d-4bc6-8b03-5cd5682aca0f None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:55.989948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda9394-532d-4bc6-8b03-5cd5682aca0f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:55.990248 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda9394-532d-4bc6-8b03-5cd5682aca0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:55.991675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda9394-532d-4bc6-8b03-5cd5682aca0f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:56.000738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 157/314] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:56.005975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3b0d5-f5d7-4320-b2d6-bd1245110805 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:56.022796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:56.119390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3b0d5-f5d7-4320-b2d6-bd1245110805 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:56.154778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbc1a3-6692-4fce-9932-2f2ee7d3f510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:56.156051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbc1a3-6692-4fce-9932-2f2ee7d3f510 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:56.156565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbc1a3-6692-4fce-9932-2f2ee7d3f510 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:56.157038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbc1a3-6692-4fce-9932-2f2ee7d3f510 None admin] PATH_INFO: `/v3/users/22c8ec1706ac451a946494eb8709ee0a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:56.158441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbc1a3-6692-4fce-9932-2f2ee7d3f510 None admin] RBAC: Authorizing `identity:get_user(user_id=22c8ec1706ac451a946494eb8709ee0a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:56.160146 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbc1a3-6692-4fce-9932-2f2ee7d3f510 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:56.162358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 158/315] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:56 2022] GET /identity/v3/users/22c8ec1706ac451a946494eb8709ee0a => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:56.167883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda9394-532d-4bc6-8b03-5cd5682aca0f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:56.268678 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda9394-532d-4bc6-8b03-5cd5682aca0f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:56.302532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d7f1c-db94-494c-8172-8a739624ac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:56.303102 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d7f1c-db94-494c-8172-8a739624ac63 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:56.303401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d7f1c-db94-494c-8172-8a739624ac63 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:56.303672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d7f1c-db94-494c-8172-8a739624ac63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:56.325367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:56.371536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d7f1c-db94-494c-8172-8a739624ac63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:56.373006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d7f1c-db94-494c-8172-8a739624ac63 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:56.470013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 158/316] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:20:56.475311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbc1a3-6692-4fce-9932-2f2ee7d3f510 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:56.555672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:56.579959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbc1a3-6692-4fce-9932-2f2ee7d3f510 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:56.614767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf8653-a9fa-499b-99dd-0d133c694de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:56.615695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf8653-a9fa-499b-99dd-0d133c694de5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:56.615982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf8653-a9fa-499b-99dd-0d133c694de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:56.616260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf8653-a9fa-499b-99dd-0d133c694de5 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:56.617552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf8653-a9fa-499b-99dd-0d133c694de5 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:56.619058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf8653-a9fa-499b-99dd-0d133c694de5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:56.621340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 159/317] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:20:56 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:56.627613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d7f1c-db94-494c-8172-8a739624ac63 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:56.686454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:56.736521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d7f1c-db94-494c-8172-8a739624ac63 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:56.773490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6249b-0c23-4d28-ac82-34316623fc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:56.774156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6249b-0c23-4d28-ac82-34316623fc7a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:56.774442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6249b-0c23-4d28-ac82-34316623fc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:56.774721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6249b-0c23-4d28-ac82-34316623fc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:56.775633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6249b-0c23-4d28-ac82-34316623fc7a None admin] RBAC: Adding query filter params (role.id=3ff68e8f3bb340bf985c55b19b58c222, scope.project.id=4b61245437cd4d14997cf3df4ea706f0, user.id=22c8ec1706ac451a946494eb8709ee0a) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:56.776533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6249b-0c23-4d28-ac82-34316623fc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:56.778020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6249b-0c23-4d28-ac82-34316623fc7a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:56.784634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 159/318] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:20:56 2022] GET /identity/v3/role_assignments?user.id=22c8ec1706ac451a946494eb8709ee0a&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=3ff68e8f3bb340bf985c55b19b58c222 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:57.563488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e1956-53f2-4291-87ce-7e61ed6a2959 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:57.563749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e1956-53f2-4291-87ce-7e61ed6a2959 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:57.564022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e1956-53f2-4291-87ce-7e61ed6a2959 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:57.565410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 160/319] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:57.571008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ac1ff-efa1-4a98-801d-b2da2eda591c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:57.571319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ac1ff-efa1-4a98-801d-b2da2eda591c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:57.571611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ac1ff-efa1-4a98-801d-b2da2eda591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:57.588746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-645ac1ff-efa1-4a98-801d-b2da2eda591c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:57.589936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-645ac1ff-efa1-4a98-801d-b2da2eda591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:57.663348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 160/320] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:57.683793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e1956-53f2-4291-87ce-7e61ed6a2959 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:57.707778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69394-61b7-4d18-91d6-ef1d84b745c2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:57.708073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69394-61b7-4d18-91d6-ef1d84b745c2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:57.708356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69394-61b7-4d18-91d6-ef1d84b745c2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:57.709900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 161/321] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:57.715714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4f670-8a8c-4dec-9b64-c960591387b6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:57.716234 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4f670-8a8c-4dec-9b64-c960591387b6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:57.716948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4f670-8a8c-4dec-9b64-c960591387b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:57.736230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c8d4f670-8a8c-4dec-9b64-c960591387b6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:57.737429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c8d4f670-8a8c-4dec-9b64-c960591387b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:57.782019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:57.796252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e1956-53f2-4291-87ce-7e61ed6a2959 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:57.808518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 162/322] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:20:57.809966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca704a-2b9a-466e-9688-2271d338fd30 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:57.810261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca704a-2b9a-466e-9688-2271d338fd30 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:57.810530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca704a-2b9a-466e-9688-2271d338fd30 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:57.811985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 163/323] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:20:57.817215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850698e-6690-4032-b195-81d1f0131037 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:57.817437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850698e-6690-4032-b195-81d1f0131037 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:57.817733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850698e-6690-4032-b195-81d1f0131037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:57.830247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cb190-ff48-496b-934a-0daf0dfc2a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:57.831032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cb190-ff48-496b-934a-0daf0dfc2a1d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:57.831323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cb190-ff48-496b-934a-0daf0dfc2a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:57.831602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cb190-ff48-496b-934a-0daf0dfc2a1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:57.833855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3850698e-6690-4032-b195-81d1f0131037 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:20:57.835050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3850698e-6690-4032-b195-81d1f0131037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:20:57.842299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cb190-ff48-496b-934a-0daf0dfc2a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ec6cb190-ff48-496b-934a-0daf0dfc2a1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:57.849900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:57.850866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:57.851722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 161/324] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:57.852382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cb190-ff48-496b-934a-0daf0dfc2a1d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:57.856458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:57.904291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 164/325] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:20:57.905558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850698e-6690-4032-b195-81d1f0131037 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:57.956115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cb190-ff48-496b-934a-0daf0dfc2a1d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:57.990890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21386344-dbad-434c-bd8d-a650c6256d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:57.991716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21386344-dbad-434c-bd8d-a650c6256d69 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:57.991995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21386344-dbad-434c-bd8d-a650c6256d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:57.992266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21386344-dbad-434c-bd8d-a650c6256d69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:58.002629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21386344-dbad-434c-bd8d-a650c6256d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-21386344-dbad-434c-bd8d-a650c6256d69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:20:58.010407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:20:58.011307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:20:58.012098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 162/326] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:20:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:20:58.012729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21386344-dbad-434c-bd8d-a650c6256d69 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:58.013352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850698e-6690-4032-b195-81d1f0131037 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:58.023579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:58.051918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4403f5-0871-466b-a831-ca9be87d4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:58.052985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4403f5-0871-466b-a831-ca9be87d4408 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:58.053264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4403f5-0871-466b-a831-ca9be87d4408 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:58.053537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4403f5-0871-466b-a831-ca9be87d4408 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:58.054282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4403f5-0871-466b-a831-ca9be87d4408 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:58.054962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4403f5-0871-466b-a831-ca9be87d4408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:58.057070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4403f5-0871-466b-a831-ca9be87d4408 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:58.065584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 165/327] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:58.066379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4403f5-0871-466b-a831-ca9be87d4408 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:58.074267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:58.117259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21386344-dbad-434c-bd8d-a650c6256d69 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:58.151912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127162a-6a74-4e79-a68b-d7ec6310df12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:58.152719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127162a-6a74-4e79-a68b-d7ec6310df12 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:58.153005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127162a-6a74-4e79-a68b-d7ec6310df12 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:58.153281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127162a-6a74-4e79-a68b-d7ec6310df12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:58.154475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127162a-6a74-4e79-a68b-d7ec6310df12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:58.155973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127162a-6a74-4e79-a68b-d7ec6310df12 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:58.157597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 163/328] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:20:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:58.162420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127162a-6a74-4e79-a68b-d7ec6310df12 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:58.169330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4403f5-0871-466b-a831-ca9be87d4408 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:58.202860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b49d9-dae6-4a63-b4a4-c0ad6e81144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:58.203556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b49d9-dae6-4a63-b4a4-c0ad6e81144f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:58.203835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b49d9-dae6-4a63-b4a4-c0ad6e81144f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:58.204103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b49d9-dae6-4a63-b4a4-c0ad6e81144f None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:58.204847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b49d9-dae6-4a63-b4a4-c0ad6e81144f None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:58.205150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b49d9-dae6-4a63-b4a4-c0ad6e81144f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:58.206529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b49d9-dae6-4a63-b4a4-c0ad6e81144f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:58.214480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 166/329] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:20:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:58.215226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b49d9-dae6-4a63-b4a4-c0ad6e81144f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:58.244472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:58.264802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127162a-6a74-4e79-a68b-d7ec6310df12 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:58.293292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:58.298792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45069bd7-9802-417f-9b3b-128bf0e42154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:58.299776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45069bd7-9802-417f-9b3b-128bf0e42154 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:58.300174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45069bd7-9802-417f-9b3b-128bf0e42154 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:58.300508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45069bd7-9802-417f-9b3b-128bf0e42154 None admin] PATH_INFO: `/v3/users/3748b0e8c8b24ac6accc84aa649df6ff` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:58.301856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45069bd7-9802-417f-9b3b-128bf0e42154 None admin] RBAC: Authorizing `identity:get_user(user_id=3748b0e8c8b24ac6accc84aa649df6ff)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:58.303457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45069bd7-9802-417f-9b3b-128bf0e42154 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:58.305377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 164/330] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:58 2022] GET /identity/v3/users/3748b0e8c8b24ac6accc84aa649df6ff => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:58.306148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45069bd7-9802-417f-9b3b-128bf0e42154 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:58.318089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b49d9-dae6-4a63-b4a4-c0ad6e81144f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:58.353591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb83671-e996-4366-a11d-4dd0d22e8603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:58.354269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb83671-e996-4366-a11d-4dd0d22e8603 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:58.354547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb83671-e996-4366-a11d-4dd0d22e8603 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:58.354815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb83671-e996-4366-a11d-4dd0d22e8603 None admin] PATH_INFO: `/v3/groups` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:58.355520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb83671-e996-4366-a11d-4dd0d22e8603 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:58.356974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb83671-e996-4366-a11d-4dd0d22e8603 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:58.371808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 167/331] 10.222.0.42 () {60 vars in 1134 bytes} [Sat Jul 2 03:20:58 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:20:58.372908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb83671-e996-4366-a11d-4dd0d22e8603 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:58.430576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45069bd7-9802-417f-9b3b-128bf0e42154 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:58.471080 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747d30-78bc-43e4-bac5-9a20b8ce0a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:58.472002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747d30-78bc-43e4-bac5-9a20b8ce0a91 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:58.472289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747d30-78bc-43e4-bac5-9a20b8ce0a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:58.472570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747d30-78bc-43e4-bac5-9a20b8ce0a91 None admin] PATH_INFO: `/v3/users/c08d14c4ed964998bc44513160b9374f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:58.473813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21747d30-78bc-43e4-bac5-9a20b8ce0a91 None admin] RBAC: Authorizing `identity:get_user(user_id=c08d14c4ed964998bc44513160b9374f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:58.475325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21747d30-78bc-43e4-bac5-9a20b8ce0a91 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:58.477042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 165/332] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:20:58 2022] GET /identity/v3/users/c08d14c4ed964998bc44513160b9374f => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:20:58.477530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:58.483450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747d30-78bc-43e4-bac5-9a20b8ce0a91 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:20:58.489902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb83671-e996-4366-a11d-4dd0d22e8603 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:58.532285 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ff91e-6900-44c3-b6e1-73ce269f703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:58.533565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ff91e-6900-44c3-b6e1-73ce269f703d None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:58.533793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ff91e-6900-44c3-b6e1-73ce269f703d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:58.534069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ff91e-6900-44c3-b6e1-73ce269f703d None admin] PATH_INFO: `/v3/system/users/3748b0e8c8b24ac6accc84aa649df6ff/roles/3ff68e8f3bb340bf985c55b19b58c222` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:58.535987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ff91e-6900-44c3-b6e1-73ce269f703d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3748b0e8c8b24ac6accc84aa649df6ff, role_id=3ff68e8f3bb340bf985c55b19b58c222)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:58.537464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ff91e-6900-44c3-b6e1-73ce269f703d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:58.547681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 168/333] 10.222.0.42 () {58 vars in 1334 bytes} [Sat Jul 2 03:20:58 2022] PUT /identity/v3/system/users/3748b0e8c8b24ac6accc84aa649df6ff/roles/3ff68e8f3bb340bf985c55b19b58c222 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:20:58.563109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:20:58.609495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747d30-78bc-43e4-bac5-9a20b8ce0a91 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:20:58.653567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b139f9-357e-4e34-b7b9-c6c8e9848acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:20:58.654241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b139f9-357e-4e34-b7b9-c6c8e9848acc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:20:58.654522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b139f9-357e-4e34-b7b9-c6c8e9848acc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:20:58.654795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b139f9-357e-4e34-b7b9-c6c8e9848acc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:20:58.655707 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b139f9-357e-4e34-b7b9-c6c8e9848acc None admin] RBAC: Adding query filter params (role.id=6aab4b825fea4f4292adc656ae17a709, user.id=c08d14c4ed964998bc44513160b9374f, scope.system=all) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:20:58.656019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b139f9-357e-4e34-b7b9-c6c8e9848acc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:20:58.657499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b139f9-357e-4e34-b7b9-c6c8e9848acc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:20:58.672616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 166/334] 10.222.0.42 () {56 vars in 1308 bytes} [Sat Jul 2 03:20:58 2022] GET /identity/v3/role_assignments?user.id=c08d14c4ed964998bc44513160b9374f&scope.system=all&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:00.899711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b3b16-4875-4b43-8239-6f02fc61bcab None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:00.901054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b3b16-4875-4b43-8239-6f02fc61bcab None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:00.901531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b3b16-4875-4b43-8239-6f02fc61bcab None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:00.903406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 169/335] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:00.908525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eed0b-fc97-41b6-be17-c5827c91efcd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:00.909358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eed0b-fc97-41b6-be17-c5827c91efcd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:00.909823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eed0b-fc97-41b6-be17-c5827c91efcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:00.936309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-353eed0b-fc97-41b6-be17-c5827c91efcd None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:00.937848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-353eed0b-fc97-41b6-be17-c5827c91efcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:01.022121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 167/336] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:01.060839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b3b16-4875-4b43-8239-6f02fc61bcab None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:01.127516 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c249ec-327f-485e-b182-dcdcd4cf3b36 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.128167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c249ec-327f-485e-b182-dcdcd4cf3b36 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.128638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c249ec-327f-485e-b182-dcdcd4cf3b36 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.130704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 168/337] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:21:01.142337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bfb89-eb21-4a0e-b0ff-8e3c286fcf47 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.142929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bfb89-eb21-4a0e-b0ff-8e3c286fcf47 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.143594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bfb89-eb21-4a0e-b0ff-8e3c286fcf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.164592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bb8bfb89-eb21-4a0e-b0ff-8e3c286fcf47 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:01.166263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bb8bfb89-eb21-4a0e-b0ff-8e3c286fcf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:01.253392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b3b16-4875-4b43-8239-6f02fc61bcab None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:01.275720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:01.307076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 169/338] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:01.328053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f1388-6c29-4bd7-b301-4e42c617073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:01.328892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1388-6c29-4bd7-b301-4e42c617073b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.329176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1388-6c29-4bd7-b301-4e42c617073b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.329453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f1388-6c29-4bd7-b301-4e42c617073b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.344893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f1388-6c29-4bd7-b301-4e42c617073b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e01f1388-6c29-4bd7-b301-4e42c617073b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:01.357463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:01.358352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:01.359166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 170/339] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:01.368354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f1388-6c29-4bd7-b301-4e42c617073b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:01.368495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bfb89-eb21-4a0e-b0ff-8e3c286fcf47 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:01.391660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:01.496852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bfb89-eb21-4a0e-b0ff-8e3c286fcf47 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:01.541665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f9046-3cea-4733-9c15-9bb373bc13ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:01.542337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f9046-3cea-4733-9c15-9bb373bc13ce None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.542616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f9046-3cea-4733-9c15-9bb373bc13ce None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.542886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f9046-3cea-4733-9c15-9bb373bc13ce None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.543655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f9046-3cea-4733-9c15-9bb373bc13ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:01.543962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f9046-3cea-4733-9c15-9bb373bc13ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:01.545332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f9046-3cea-4733-9c15-9bb373bc13ce None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:01.558516 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 170/340] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:01.559644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3af237-d49f-45e4-bf39-ce455826fb16 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.559929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3af237-d49f-45e4-bf39-ce455826fb16 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.560200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3af237-d49f-45e4-bf39-ce455826fb16 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.570622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 171/341] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:01.571550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3af237-d49f-45e4-bf39-ce455826fb16 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:01.594408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f1388-6c29-4bd7-b301-4e42c617073b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:01.655690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:01.666601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d48bcf-08de-4fd1-a53c-f9d68f6b9c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:01.667449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d48bcf-08de-4fd1-a53c-f9d68f6b9c44 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.667730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d48bcf-08de-4fd1-a53c-f9d68f6b9c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.668001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d48bcf-08de-4fd1-a53c-f9d68f6b9c44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.691594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d48bcf-08de-4fd1-a53c-f9d68f6b9c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d0d48bcf-08de-4fd1-a53c-f9d68f6b9c44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:01.699433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:01.700300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 02 03:21:01.701085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:01.709647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 171/342] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:01.713320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2364c0-4354-4951-9dd1-eb513c0937c1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.713609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2364c0-4354-4951-9dd1-eb513c0937c1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.714027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2364c0-4354-4951-9dd1-eb513c0937c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.736017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9e2364c0-4354-4951-9dd1-eb513c0937c1 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:01.737294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e2364c0-4354-4951-9dd1-eb513c0937c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:01.744323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3af237-d49f-45e4-bf39-ce455826fb16 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:01.786170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3d976-f1b0-49e5-a73d-5e559eb05412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:01.786741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3d976-f1b0-49e5-a73d-5e559eb05412 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.787019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3d976-f1b0-49e5-a73d-5e559eb05412 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.787317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3d976-f1b0-49e5-a73d-5e559eb05412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.810000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 172/343] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:01.810649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2364c0-4354-4951-9dd1-eb513c0937c1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:01.835523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:01.863113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3d976-f1b0-49e5-a73d-5e559eb05412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:01.866886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3d976-f1b0-49e5-a73d-5e559eb05412 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:01.917485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:01.922396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2364c0-4354-4951-9dd1-eb513c0937c1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:01.958918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5e1b1-ccb3-4c3d-ba7d-7fe446fa1ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:01.959629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5e1b1-ccb3-4c3d-ba7d-7fe446fa1ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.959910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5e1b1-ccb3-4c3d-ba7d-7fe446fa1ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.960279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5e1b1-ccb3-4c3d-ba7d-7fe446fa1ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.961029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5e1b1-ccb3-4c3d-ba7d-7fe446fa1ab4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:01.961332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5e1b1-ccb3-4c3d-ba7d-7fe446fa1ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:01.962745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5e1b1-ccb3-4c3d-ba7d-7fe446fa1ab4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:01.972912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 172/344] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:01.974029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f1cb8-a4be-4b31-9c59-9ec5147448f8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:01.974173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 173/345] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:21:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:01.974363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f1cb8-a4be-4b31-9c59-9ec5147448f8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:01.974579 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f1cb8-a4be-4b31-9c59-9ec5147448f8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:01.974949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5e1b1-ccb3-4c3d-ba7d-7fe446fa1ab4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:01.976094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 173/346] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:01.978062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f1cb8-a4be-4b31-9c59-9ec5147448f8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:02.063858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:02.086714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f1cb8-a4be-4b31-9c59-9ec5147448f8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:02.092989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5e1b1-ccb3-4c3d-ba7d-7fe446fa1ab4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:02.123140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c4ca1-3ea4-403b-b33d-6b068661949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:02.124049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c4ca1-3ea4-403b-b33d-6b068661949b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.124308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c4ca1-3ea4-403b-b33d-6b068661949b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.124578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c4ca1-3ea4-403b-b33d-6b068661949b None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.125758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c4ca1-3ea4-403b-b33d-6b068661949b None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:02.127221 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c4ca1-3ea4-403b-b33d-6b068661949b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:02.130749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 174/347] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:01 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:02.132009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be530d1-b018-4426-a9f6-4fc77340fa1b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.132286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be530d1-b018-4426-a9f6-4fc77340fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.132558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be530d1-b018-4426-a9f6-4fc77340fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.146111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebb099-acb7-44be-b014-58c5f3f5c0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:02.146926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebb099-acb7-44be-b014-58c5f3f5c0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.147205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebb099-acb7-44be-b014-58c5f3f5c0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.147507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebb099-acb7-44be-b014-58c5f3f5c0a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.153482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5be530d1-b018-4426-a9f6-4fc77340fa1b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:02.155211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5be530d1-b018-4426-a9f6-4fc77340fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:02.161675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebb099-acb7-44be-b014-58c5f3f5c0a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:02.168399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f3ebb099-acb7-44be-b014-58c5f3f5c0a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:02.171923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:02.172867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:02.173712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 174/348] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:02.174364 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebb099-acb7-44be-b014-58c5f3f5c0a1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:02.253156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 175/349] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:02.254018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be530d1-b018-4426-a9f6-4fc77340fa1b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:02.298837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebb099-acb7-44be-b014-58c5f3f5c0a1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:02.331554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:02.350650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2c221-8b4d-414c-b9e0-1ab2383f7c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:02.352368 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2c221-8b4d-414c-b9e0-1ab2383f7c41 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.352845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2c221-8b4d-414c-b9e0-1ab2383f7c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.353724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2c221-8b4d-414c-b9e0-1ab2383f7c41 None admin] PATH_INFO: `/v3/users/3748b0e8c8b24ac6accc84aa649df6ff` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.355181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2c221-8b4d-414c-b9e0-1ab2383f7c41 None admin] RBAC: Authorizing `identity:get_user(user_id=3748b0e8c8b24ac6accc84aa649df6ff)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:02.356998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2c221-8b4d-414c-b9e0-1ab2383f7c41 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:02.358599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 175/350] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:21:02 2022] GET /identity/v3/users/3748b0e8c8b24ac6accc84aa649df6ff => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:02.359421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2c221-8b4d-414c-b9e0-1ab2383f7c41 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:02.378832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be530d1-b018-4426-a9f6-4fc77340fa1b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:02.422882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc2908-0e59-493c-9531-9b9e3980458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:02.424078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc2908-0e59-493c-9531-9b9e3980458a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.424364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc2908-0e59-493c-9531-9b9e3980458a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.424642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc2908-0e59-493c-9531-9b9e3980458a None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.432319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc2908-0e59-493c-9531-9b9e3980458a None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:02.434406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc2908-0e59-493c-9531-9b9e3980458a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:02.436753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 176/351] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:02 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:02.439502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40665d29-a827-4ee7-b622-8601cc66f8fa None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.439814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40665d29-a827-4ee7-b622-8601cc66f8fa None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.440090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40665d29-a827-4ee7-b622-8601cc66f8fa None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.441404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 177/352] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:02.442107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40665d29-a827-4ee7-b622-8601cc66f8fa None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:02.457396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:02.506252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2c221-8b4d-414c-b9e0-1ab2383f7c41 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:02.553241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320c39d-7e77-4e67-890c-81315e5a3bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:02.554459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320c39d-7e77-4e67-890c-81315e5a3bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.555086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320c39d-7e77-4e67-890c-81315e5a3bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.555595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320c39d-7e77-4e67-890c-81315e5a3bc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.557250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320c39d-7e77-4e67-890c-81315e5a3bc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:02.558222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320c39d-7e77-4e67-890c-81315e5a3bc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:02.560402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320c39d-7e77-4e67-890c-81315e5a3bc9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:02.575886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 176/353] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:21:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:02.575886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320c39d-7e77-4e67-890c-81315e5a3bc9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:02.579726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40665d29-a827-4ee7-b622-8601cc66f8fa None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:02.621739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7cf2-137d-476a-a088-6f51307eb3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:02.622470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7cf2-137d-476a-a088-6f51307eb3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.623162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7cf2-137d-476a-a088-6f51307eb3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.623484 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7cf2-137d-476a-a088-6f51307eb3a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.637972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b7cf2-137d-476a-a088-6f51307eb3a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:02.639590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6d4b7cf2-137d-476a-a088-6f51307eb3a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:02.646968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:02.647936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:02.648738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 178/354] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:02.649372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7cf2-137d-476a-a088-6f51307eb3a2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:02.702361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320c39d-7e77-4e67-890c-81315e5a3bc9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:02.733809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:02.751254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a0c68-f574-47dd-9829-67a279141bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:02.751997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a0c68-f574-47dd-9829-67a279141bfc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.753339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a0c68-f574-47dd-9829-67a279141bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.753339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a0c68-f574-47dd-9829-67a279141bfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.753499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a0c68-f574-47dd-9829-67a279141bfc None admin] RBAC: Adding query filter params (role.id=3ff68e8f3bb340bf985c55b19b58c222, user.id=3748b0e8c8b24ac6accc84aa649df6ff, scope.system=all) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:02.753674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a0c68-f574-47dd-9829-67a279141bfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:02.755536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a0c68-f574-47dd-9829-67a279141bfc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:02.768314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 177/355] 10.222.0.42 () {56 vars in 1308 bytes} [Sat Jul 2 03:21:02 2022] GET /identity/v3/role_assignments?user.id=3748b0e8c8b24ac6accc84aa649df6ff&scope.system=all&role.id=3ff68e8f3bb340bf985c55b19b58c222 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:02.769415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c44b-ef20-4297-bf09-c52058ef8833 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.769701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c44b-ef20-4297-bf09-c52058ef8833 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.769979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c44b-ef20-4297-bf09-c52058ef8833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.772395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7cf2-137d-476a-a088-6f51307eb3a2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:02.798337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3935c44b-ef20-4297-bf09-c52058ef8833 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:02.799601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3935c44b-ef20-4297-bf09-c52058ef8833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:02.825102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6b3c7-7282-406a-a16d-578a62c25591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:02.825777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6b3c7-7282-406a-a16d-578a62c25591 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:02.826222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6b3c7-7282-406a-a16d-578a62c25591 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:02.826505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6b3c7-7282-406a-a16d-578a62c25591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:02.830224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6b3c7-7282-406a-a16d-578a62c25591 None admin] RBAC: Adding query filter params (group.id=c676772b961f496690b42a8bb709ca64, role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=94c3ce30be2e489e9210838c8132fd92) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:02.830224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6b3c7-7282-406a-a16d-578a62c25591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:02.830224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6b3c7-7282-406a-a16d-578a62c25591 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:02.840129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 179/356] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:02 2022] GET /identity/v3/role_assignments?group.id=c676772b961f496690b42a8bb709ca64&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=62ac4326d6d742c5b6684746953362b1 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:02.841088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6b3c7-7282-406a-a16d-578a62c25591 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:02.913842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 178/357] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:02.915079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c44b-ef20-4297-bf09-c52058ef8833 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:02.991059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:03.005432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6b3c7-7282-406a-a16d-578a62c25591 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:03.050181 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9443d3e-0891-4709-ab9f-256877d71b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:03.051438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9443d3e-0891-4709-ab9f-256877d71b70 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.052340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9443d3e-0891-4709-ab9f-256877d71b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.052931 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9443d3e-0891-4709-ab9f-256877d71b70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:03.058198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:03.067031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c44b-ef20-4297-bf09-c52058ef8833 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:03.108259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa2e34-f388-4ed7-98e4-8a3c1c25d361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:03.109784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2e34-f388-4ed7-98e4-8a3c1c25d361 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.109784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2e34-f388-4ed7-98e4-8a3c1c25d361 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.109784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa2e34-f388-4ed7-98e4-8a3c1c25d361 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:03.110350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2e34-f388-4ed7-98e4-8a3c1c25d361 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:03.110660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2e34-f388-4ed7-98e4-8a3c1c25d361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:03.112209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa2e34-f388-4ed7-98e4-8a3c1c25d361 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:03.122446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 179/358] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:21:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:03.124402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c8636-9620-440c-a775-24f04232d915 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.125025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c8636-9620-440c-a775-24f04232d915 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.125524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c8636-9620-440c-a775-24f04232d915 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:03.134372 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 180/359] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:03.135636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c8636-9620-440c-a775-24f04232d915 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:03.152927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9443d3e-0891-4709-ab9f-256877d71b70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:03.154752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9443d3e-0891-4709-ab9f-256877d71b70 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:03.268186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c8636-9620-440c-a775-24f04232d915 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:03.275595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:03.294157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 180/360] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:03.295608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9443d3e-0891-4709-ab9f-256877d71b70 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:03.313927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62a90f-8f85-450a-9486-022125143670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:03.314150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a90f-8f85-450a-9486-022125143670 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.314393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a90f-8f85-450a-9486-022125143670 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.314556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a90f-8f85-450a-9486-022125143670 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:03.331193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62a90f-8f85-450a-9486-022125143670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:03.340911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1b62a90f-8f85-450a-9486-022125143670 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:03.342399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:03.343411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:03.344291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:03.345672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 181/361] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:03.346466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18215965-5c26-4ee8-9369-d7e642352dc8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.346749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18215965-5c26-4ee8-9369-d7e642352dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.347023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18215965-5c26-4ee8-9369-d7e642352dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:03.376399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-18215965-5c26-4ee8-9369-d7e642352dc8 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:03.377623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-18215965-5c26-4ee8-9369-d7e642352dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:03.434641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9443d3e-0891-4709-ab9f-256877d71b70 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:03.457668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 182/362] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:03.458704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18215965-5c26-4ee8-9369-d7e642352dc8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:03.477864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf818f04-2ee5-447c-93b4-3eec7175fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:03.478628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf818f04-2ee5-447c-93b4-3eec7175fac7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.478975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf818f04-2ee5-447c-93b4-3eec7175fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.479252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf818f04-2ee5-447c-93b4-3eec7175fac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:03.567817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:03.570463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf818f04-2ee5-447c-93b4-3eec7175fac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:03.572206 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf818f04-2ee5-447c-93b4-3eec7175fac7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:03.576674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18215965-5c26-4ee8-9369-d7e642352dc8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:03.613896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:03.630504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332759e-3ed5-4a7c-aa8e-1394fc3ae144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:03.632620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332759e-3ed5-4a7c-aa8e-1394fc3ae144 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.632620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332759e-3ed5-4a7c-aa8e-1394fc3ae144 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.632620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332759e-3ed5-4a7c-aa8e-1394fc3ae144 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:03.636526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332759e-3ed5-4a7c-aa8e-1394fc3ae144 None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:03.637903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332759e-3ed5-4a7c-aa8e-1394fc3ae144 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:03.640046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 183/363] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:03 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:03.640823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332759e-3ed5-4a7c-aa8e-1394fc3ae144 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:03.692690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 181/364] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:03.693967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf818f04-2ee5-447c-93b4-3eec7175fac7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:03.782809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332759e-3ed5-4a7c-aa8e-1394fc3ae144 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:03.812345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf818f04-2ee5-447c-93b4-3eec7175fac7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:03.827956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46935d-7700-4834-9167-79b7bb0c5a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:03.828568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46935d-7700-4834-9167-79b7bb0c5a56 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.828848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46935d-7700-4834-9167-79b7bb0c5a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.829125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c46935d-7700-4834-9167-79b7bb0c5a56 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:03.829903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46935d-7700-4834-9167-79b7bb0c5a56 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:03.830195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46935d-7700-4834-9167-79b7bb0c5a56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:03.831602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c46935d-7700-4834-9167-79b7bb0c5a56 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:03.841969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 184/365] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:03.843098 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46935d-7700-4834-9167-79b7bb0c5a56 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:03.852458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92397d8-21f5-46e2-bb09-92718ea0079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:03.853940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92397d8-21f5-46e2-bb09-92718ea0079b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.854503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92397d8-21f5-46e2-bb09-92718ea0079b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.854964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92397d8-21f5-46e2-bb09-92718ea0079b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:03.858589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:03.868524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92397d8-21f5-46e2-bb09-92718ea0079b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:03.875705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f92397d8-21f5-46e2-bb09-92718ea0079b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:03.877478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:03.879709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 182/366] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:03.881374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92397d8-21f5-46e2-bb09-92718ea0079b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:03.960908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c46935d-7700-4834-9167-79b7bb0c5a56 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:03.998057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb452f-462f-4b45-ac24-856a6b86bc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:03.998732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92397d8-21f5-46e2-bb09-92718ea0079b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:03.998928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb452f-462f-4b45-ac24-856a6b86bc96 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:03.999205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb452f-462f-4b45-ac24-856a6b86bc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:03.999509 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb452f-462f-4b45-ac24-856a6b86bc96 None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:04.000731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb452f-462f-4b45-ac24-856a6b86bc96 None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:04.002193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb452f-462f-4b45-ac24-856a6b86bc96 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:04.006570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 185/367] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:03 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:04.006570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb452f-462f-4b45-ac24-856a6b86bc96 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:04.039440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a38900-6792-4f7a-a17a-319dc16ffca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:04.040345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a38900-6792-4f7a-a17a-319dc16ffca8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:04.040632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a38900-6792-4f7a-a17a-319dc16ffca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:04.040940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a38900-6792-4f7a-a17a-319dc16ffca8 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:04.042309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a38900-6792-4f7a-a17a-319dc16ffca8 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:04.044613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a38900-6792-4f7a-a17a-319dc16ffca8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:04.045704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 183/368] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:03 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:04.047487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a38900-6792-4f7a-a17a-319dc16ffca8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:04.124957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:04.146394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:04.155126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb452f-462f-4b45-ac24-856a6b86bc96 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:04.180211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a38900-6792-4f7a-a17a-319dc16ffca8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:04.196783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcccd9-c2f3-4227-95c9-628c938ac9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:04.197566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcccd9-c2f3-4227-95c9-628c938ac9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:04.197844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcccd9-c2f3-4227-95c9-628c938ac9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:04.198119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcccd9-c2f3-4227-95c9-628c938ac9d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:04.223322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b208c0-39ab-43e2-beaf-5a97e85db0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:04.224086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b208c0-39ab-43e2-beaf-5a97e85db0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:04.224372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b208c0-39ab-43e2-beaf-5a97e85db0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:04.224647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b208c0-39ab-43e2-beaf-5a97e85db0d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:04.225397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b208c0-39ab-43e2-beaf-5a97e85db0d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:04.225814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b208c0-39ab-43e2-beaf-5a97e85db0d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:04.227367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b208c0-39ab-43e2-beaf-5a97e85db0d3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:04.237074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 184/369] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:04.237719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b208c0-39ab-43e2-beaf-5a97e85db0d3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:04.285185 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcccd9-c2f3-4227-95c9-628c938ac9d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:04.285718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcccd9-c2f3-4227-95c9-628c938ac9d9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:04.358029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b208c0-39ab-43e2-beaf-5a97e85db0d3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:04.389592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:04.406223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:04.406758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4ed97-606b-4319-a43a-bbc52fddfad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:04.407463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4ed97-606b-4319-a43a-bbc52fddfad8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:04.407752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4ed97-606b-4319-a43a-bbc52fddfad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:04.408031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4ed97-606b-4319-a43a-bbc52fddfad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:04.408867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4ed97-606b-4319-a43a-bbc52fddfad8 None admin] RBAC: Adding query filter params (group.id=c676772b961f496690b42a8bb709ca64, role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=94c3ce30be2e489e9210838c8132fd92) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:04.409178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4ed97-606b-4319-a43a-bbc52fddfad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:04.410672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4ed97-606b-4319-a43a-bbc52fddfad8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:04.416179 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 186/370] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:04.417271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcccd9-c2f3-4227-95c9-628c938ac9d9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:04.423373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 185/371] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:04 2022] GET /identity/v3/role_assignments?group.id=c676772b961f496690b42a8bb709ca64&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:04.423373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4ed97-606b-4319-a43a-bbc52fddfad8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:04.552333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcccd9-c2f3-4227-95c9-628c938ac9d9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:04.586763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4ed97-606b-4319-a43a-bbc52fddfad8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:04.603435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40b26-598a-4fa1-a825-4a1c8f9795bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:04.604526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40b26-598a-4fa1-a825-4a1c8f9795bb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:04.606829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40b26-598a-4fa1-a825-4a1c8f9795bb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:04.606829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa40b26-598a-4fa1-a825-4a1c8f9795bb None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:04.608849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa40b26-598a-4fa1-a825-4a1c8f9795bb None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:04.610636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa40b26-598a-4fa1-a825-4a1c8f9795bb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:04.613214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 187/372] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:04 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:04.614031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40b26-598a-4fa1-a825-4a1c8f9795bb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:04.643901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9e74-5e6a-4566-9270-8fc0e98449d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:04.644929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9e74-5e6a-4566-9270-8fc0e98449d1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:04.645429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9e74-5e6a-4566-9270-8fc0e98449d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:04.645770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9e74-5e6a-4566-9270-8fc0e98449d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:04.708046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:04.723389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:04.741058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b9e74-5e6a-4566-9270-8fc0e98449d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:04.743511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b9e74-5e6a-4566-9270-8fc0e98449d1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:04.755334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa40b26-598a-4fa1-a825-4a1c8f9795bb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:04.805857 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437c6f-32af-46c8-833f-3778efba639f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:04.807574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437c6f-32af-46c8-833f-3778efba639f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:04.808455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437c6f-32af-46c8-833f-3778efba639f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:04.809438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a437c6f-32af-46c8-833f-3778efba639f None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:04.811448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a437c6f-32af-46c8-833f-3778efba639f None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:04.813963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a437c6f-32af-46c8-833f-3778efba639f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:04.817210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 188/373] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:04 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:04.818864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437c6f-32af-46c8-833f-3778efba639f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:04.875657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 186/374] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:04.876585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9e74-5e6a-4566-9270-8fc0e98449d1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:04.943817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a437c6f-32af-46c8-833f-3778efba639f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:04.970782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:04.976926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:04.981025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee6eac-ba4f-43e1-b227-e352cf1328e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:04.981969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee6eac-ba4f-43e1-b227-e352cf1328e9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:04.982446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee6eac-ba4f-43e1-b227-e352cf1328e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:04.982919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee6eac-ba4f-43e1-b227-e352cf1328e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:04.983980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee6eac-ba4f-43e1-b227-e352cf1328e9 None admin] RBAC: Adding query filter params (group.id=c676772b961f496690b42a8bb709ca64, role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=4b61245437cd4d14997cf3df4ea706f0) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:04.984506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee6eac-ba4f-43e1-b227-e352cf1328e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:04.985249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9e74-5e6a-4566-9270-8fc0e98449d1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:04.986391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee6eac-ba4f-43e1-b227-e352cf1328e9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:04.992887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 189/375] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:04 2022] GET /identity/v3/role_assignments?group.id=c676772b961f496690b42a8bb709ca64&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:04.994096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee6eac-ba4f-43e1-b227-e352cf1328e9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:05.033355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5877bf6-7cf1-4661-bf59-5858680f427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:05.034253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5877bf6-7cf1-4661-bf59-5858680f427c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.034531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5877bf6-7cf1-4661-bf59-5858680f427c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.034804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5877bf6-7cf1-4661-bf59-5858680f427c None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:05.036149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5877bf6-7cf1-4661-bf59-5858680f427c None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:05.037654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5877bf6-7cf1-4661-bf59-5858680f427c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:05.040031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 187/376] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:04 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:05.041510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1736f-db3e-4f6e-a770-c9d035f46600 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.041736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1736f-db3e-4f6e-a770-c9d035f46600 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.042017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1736f-db3e-4f6e-a770-c9d035f46600 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:05.049603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 188/377] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:05.049603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1736f-db3e-4f6e-a770-c9d035f46600 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:05.111206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee6eac-ba4f-43e1-b227-e352cf1328e9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:05.148450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3c6bc-c62f-4260-bba5-6cc9a1ad0506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:05.149333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3c6bc-c62f-4260-bba5-6cc9a1ad0506 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.149618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3c6bc-c62f-4260-bba5-6cc9a1ad0506 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.149915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3c6bc-c62f-4260-bba5-6cc9a1ad0506 None admin] PATH_INFO: `/v3/projects/08e65693c4ea4b43b13797fe93e3a44a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:05.151402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3c6bc-c62f-4260-bba5-6cc9a1ad0506 None admin] RBAC: Authorizing `identity:get_project(project_id=08e65693c4ea4b43b13797fe93e3a44a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:05.152893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3c6bc-c62f-4260-bba5-6cc9a1ad0506 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:05.154862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 190/378] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:04 2022] GET /identity/v3/projects/08e65693c4ea4b43b13797fe93e3a44a => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:05.159418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ae6d8-3bb1-41de-b9f4-6f3d0b3f41fc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.159418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ae6d8-3bb1-41de-b9f4-6f3d0b3f41fc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.159418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ae6d8-3bb1-41de-b9f4-6f3d0b3f41fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:05.179904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1e7ae6d8-3bb1-41de-b9f4-6f3d0b3f41fc None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:05.181333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1e7ae6d8-3bb1-41de-b9f4-6f3d0b3f41fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:05.199369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1736f-db3e-4f6e-a770-c9d035f46600 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:05.244897 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a1b5-a3a9-4192-ba19-11bd6b3dfbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:05.245804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a1b5-a3a9-4192-ba19-11bd6b3dfbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.246466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a1b5-a3a9-4192-ba19-11bd6b3dfbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.246923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a1b5-a3a9-4192-ba19-11bd6b3dfbe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:05.248320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57a1b5-a3a9-4192-ba19-11bd6b3dfbe9 None admin] RBAC: Adding query filter params (group.id=c676772b961f496690b42a8bb709ca64, role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=4b61245437cd4d14997cf3df4ea706f0) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:05.248779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57a1b5-a3a9-4192-ba19-11bd6b3dfbe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:05.250927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57a1b5-a3a9-4192-ba19-11bd6b3dfbe9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:05.254958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:05.267524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 189/379] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:05 2022] GET /identity/v3/role_assignments?group.id=c676772b961f496690b42a8bb709ca64&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=62ac4326d6d742c5b6684746953362b1 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:05.268692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a1b5-a3a9-4192-ba19-11bd6b3dfbe9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:05.271287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 191/380] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:05.298591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:05.306092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ae6d8-3bb1-41de-b9f4-6f3d0b3f41fc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:05.441609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a1b5-a3a9-4192-ba19-11bd6b3dfbe9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:05.488891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f25bb9-9278-4300-8f47-bb480846f42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:05.489782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f25bb9-9278-4300-8f47-bb480846f42a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.490068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f25bb9-9278-4300-8f47-bb480846f42a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.490343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f25bb9-9278-4300-8f47-bb480846f42a None admin] PATH_INFO: `/v3/groups/060862c82dcd43e5a038b1428e761b90` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:05.496583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f25bb9-9278-4300-8f47-bb480846f42a None admin] RBAC: Authorizing `identity:get_group(group_id=060862c82dcd43e5a038b1428e761b90)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:05.498544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f25bb9-9278-4300-8f47-bb480846f42a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:05.500983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 190/381] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:05 2022] GET /identity/v3/groups/060862c82dcd43e5a038b1428e761b90 => generated 229 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:05.507541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f25bb9-9278-4300-8f47-bb480846f42a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:05.509252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ae6d8-3bb1-41de-b9f4-6f3d0b3f41fc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:05.587945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a13c2-cba1-41b2-88d4-b7edacdbd2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:05.589311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a13c2-cba1-41b2-88d4-b7edacdbd2ae None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.589596 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a13c2-cba1-41b2-88d4-b7edacdbd2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.589874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a13c2-cba1-41b2-88d4-b7edacdbd2ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:05.609908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a13c2-cba1-41b2-88d4-b7edacdbd2ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-aa4a13c2-cba1-41b2-88d4-b7edacdbd2ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:05.626104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:05.627020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:05.627847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 192/382] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:05.633021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a13c2-cba1-41b2-88d4-b7edacdbd2ae None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:05.644301 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:05.665488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f25bb9-9278-4300-8f47-bb480846f42a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:05.714457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:05.715957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024da0f1-6912-4e84-948f-fcc7f2641b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:05.716720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024da0f1-6912-4e84-948f-fcc7f2641b6c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.717027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024da0f1-6912-4e84-948f-fcc7f2641b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.717315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024da0f1-6912-4e84-948f-fcc7f2641b6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:05.718165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024da0f1-6912-4e84-948f-fcc7f2641b6c None admin] RBAC: Adding query filter params (group.id=060862c82dcd43e5a038b1428e761b90, role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=08e65693c4ea4b43b13797fe93e3a44a) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:05.718515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024da0f1-6912-4e84-948f-fcc7f2641b6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:05.720047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024da0f1-6912-4e84-948f-fcc7f2641b6c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:05.735142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 191/383] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:05 2022] GET /identity/v3/role_assignments?group.id=060862c82dcd43e5a038b1428e761b90&scope.project.id=08e65693c4ea4b43b13797fe93e3a44a&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:05.767635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a13c2-cba1-41b2-88d4-b7edacdbd2ae None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:05.803141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f346deb-a12e-48da-bb12-f23a8cb36c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:05.803795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f346deb-a12e-48da-bb12-f23a8cb36c9f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.804078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f346deb-a12e-48da-bb12-f23a8cb36c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.804355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f346deb-a12e-48da-bb12-f23a8cb36c9f None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:05.805120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f346deb-a12e-48da-bb12-f23a8cb36c9f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:05.805429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f346deb-a12e-48da-bb12-f23a8cb36c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:05.807275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f346deb-a12e-48da-bb12-f23a8cb36c9f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:05.816677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 193/384] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:05.824428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024da0f1-6912-4e84-948f-fcc7f2641b6c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:05.963257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024da0f1-6912-4e84-948f-fcc7f2641b6c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:05.997528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a576cc-599f-4bdb-ba37-467783a0a5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:05.998391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a576cc-599f-4bdb-ba37-467783a0a5ea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:05.998670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a576cc-599f-4bdb-ba37-467783a0a5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:05.998942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a576cc-599f-4bdb-ba37-467783a0a5ea None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:06.000144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a576cc-599f-4bdb-ba37-467783a0a5ea None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:06.001590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a576cc-599f-4bdb-ba37-467783a0a5ea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:06.003549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 192/385] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:05 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:06.008908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f346deb-a12e-48da-bb12-f23a8cb36c9f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:06.166886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f346deb-a12e-48da-bb12-f23a8cb36c9f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:06.187768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:06.220505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e86bce-8634-4d08-8c2d-d480efc66022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:06.221442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e86bce-8634-4d08-8c2d-d480efc66022 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:06.221721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e86bce-8634-4d08-8c2d-d480efc66022 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:06.222039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e86bce-8634-4d08-8c2d-d480efc66022 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:06.223242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e86bce-8634-4d08-8c2d-d480efc66022 None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:06.224721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e86bce-8634-4d08-8c2d-d480efc66022 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:06.226451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 194/386] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:06 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:06.232449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a576cc-599f-4bdb-ba37-467783a0a5ea None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:06.258887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:06.382677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a576cc-599f-4bdb-ba37-467783a0a5ea None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:06.430323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d1848-0958-45d7-83bc-9aa8cd7bc58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:06.431218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d1848-0958-45d7-83bc-9aa8cd7bc58a None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:06.431895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d1848-0958-45d7-83bc-9aa8cd7bc58a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:06.432764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d1848-0958-45d7-83bc-9aa8cd7bc58a None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92/groups/c676772b961f496690b42a8bb709ca64/roles/62ac4326d6d742c5b6684746953362b1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:06.441404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d1848-0958-45d7-83bc-9aa8cd7bc58a None admin] RBAC: Authorizing `identity:create_grant(project_id=94c3ce30be2e489e9210838c8132fd92, group_id=c676772b961f496690b42a8bb709ca64, role_id=62ac4326d6d742c5b6684746953362b1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:06.442883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d1848-0958-45d7-83bc-9aa8cd7bc58a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:06.473833 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 193/387] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Jul 2 03:21:06 2022] PUT /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92/groups/c676772b961f496690b42a8bb709ca64/roles/62ac4326d6d742c5b6684746953362b1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:06.644971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477619b-de19-4874-9ab3-8e8f3846c02f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:06.645315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477619b-de19-4874-9ab3-8e8f3846c02f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:06.645424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477619b-de19-4874-9ab3-8e8f3846c02f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:06.646953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 195/388] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:06.652978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0835eed-d6dc-4013-8b3c-314692fd6411 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:06.653265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0835eed-d6dc-4013-8b3c-314692fd6411 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:06.653552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0835eed-d6dc-4013-8b3c-314692fd6411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:06.680556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f0835eed-d6dc-4013-8b3c-314692fd6411 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:06.682574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f0835eed-d6dc-4013-8b3c-314692fd6411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:06.765666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 194/389] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:06.788414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477619b-de19-4874-9ab3-8e8f3846c02f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:06.908464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477619b-de19-4874-9ab3-8e8f3846c02f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:06.980965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b200-8b41-43ff-855a-5562d7a720fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:06.981791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b200-8b41-43ff-855a-5562d7a720fc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:06.982073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b200-8b41-43ff-855a-5562d7a720fc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:06.982356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b200-8b41-43ff-855a-5562d7a720fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:06.999281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c43b200-8b41-43ff-855a-5562d7a720fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3c43b200-8b41-43ff-855a-5562d7a720fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:07.007311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:07.008182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:07.008979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:07.011785 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 196/390] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:07.017856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0835eed-d6dc-4013-8b3c-314692fd6411 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:07.026894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:07.133236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0835eed-d6dc-4013-8b3c-314692fd6411 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:07.184626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a74c8-072d-4ecc-b1df-821a6a344879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:07.185211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a74c8-072d-4ecc-b1df-821a6a344879 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.185489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a74c8-072d-4ecc-b1df-821a6a344879 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.185760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a74c8-072d-4ecc-b1df-821a6a344879 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.186564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a74c8-072d-4ecc-b1df-821a6a344879 None admin] RBAC: Adding query filter params (name=member) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:07.186868 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a74c8-072d-4ecc-b1df-821a6a344879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:07.188344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a74c8-072d-4ecc-b1df-821a6a344879 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:07.202987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 195/391] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:21:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:07.208777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b200-8b41-43ff-855a-5562d7a720fc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:07.266538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:07.345892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b200-8b41-43ff-855a-5562d7a720fc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:07.381623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ba261-327d-43c0-9d7e-24c03a2acaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:07.382483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ba261-327d-43c0-9d7e-24c03a2acaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.382760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ba261-327d-43c0-9d7e-24c03a2acaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.383029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ba261-327d-43c0-9d7e-24c03a2acaa7 None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.384217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ba261-327d-43c0-9d7e-24c03a2acaa7 None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:07.385941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ba261-327d-43c0-9d7e-24c03a2acaa7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:07.387917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 197/392] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:07 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:07.393531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a74c8-072d-4ecc-b1df-821a6a344879 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:07.430924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831dc0b-16b9-4ff4-a3bc-599bc4b72066 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.431235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831dc0b-16b9-4ff4-a3bc-599bc4b72066 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.431897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831dc0b-16b9-4ff4-a3bc-599bc4b72066 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.433605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 198/393] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:21:07.438773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0763142-5129-4f4d-88f7-17fec15a601e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.438996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0763142-5129-4f4d-88f7-17fec15a601e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.439274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0763142-5129-4f4d-88f7-17fec15a601e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.465397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f0763142-5129-4f4d-88f7-17fec15a601e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:07.467786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f0763142-5129-4f4d-88f7-17fec15a601e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:07.523139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:07.557920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 199/394] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:07.579050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0763142-5129-4f4d-88f7-17fec15a601e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:07.600265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a74c8-072d-4ecc-b1df-821a6a344879 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:07.639466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:07.692102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b855b7-a4bf-4c5f-8ade-39750fa440d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:07.692905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b855b7-a4bf-4c5f-8ade-39750fa440d8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.693197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b855b7-a4bf-4c5f-8ade-39750fa440d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.693483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b855b7-a4bf-4c5f-8ade-39750fa440d8 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.694742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b855b7-a4bf-4c5f-8ade-39750fa440d8 None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:07.696240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b855b7-a4bf-4c5f-8ade-39750fa440d8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:07.698342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 196/395] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:07 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:07.703682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b855b7-a4bf-4c5f-8ade-39750fa440d8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:07.707079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0763142-5129-4f4d-88f7-17fec15a601e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:07.747926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a263de-2f3d-48c0-84d9-1bc757bd9042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:07.749663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a263de-2f3d-48c0-84d9-1bc757bd9042 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.750340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a263de-2f3d-48c0-84d9-1bc757bd9042 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.753385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a263de-2f3d-48c0-84d9-1bc757bd9042 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.765296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a263de-2f3d-48c0-84d9-1bc757bd9042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c3a263de-2f3d-48c0-84d9-1bc757bd9042 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:07.773657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:07.774533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:07.776297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:07.778715 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 200/396] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:07.784062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a263de-2f3d-48c0-84d9-1bc757bd9042 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:07.874296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b855b7-a4bf-4c5f-8ade-39750fa440d8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:07.906147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:07.909316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a263de-2f3d-48c0-84d9-1bc757bd9042 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:07.913384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163cb56-fdc3-4653-b119-960835f71380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:07.914279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163cb56-fdc3-4653-b119-960835f71380 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.914560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163cb56-fdc3-4653-b119-960835f71380 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.914833 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163cb56-fdc3-4653-b119-960835f71380 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92/groups/c676772b961f496690b42a8bb709ca64/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.917254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163cb56-fdc3-4653-b119-960835f71380 None admin] RBAC: Authorizing `identity:create_grant(project_id=94c3ce30be2e489e9210838c8132fd92, group_id=c676772b961f496690b42a8bb709ca64, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:07.918716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163cb56-fdc3-4653-b119-960835f71380 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:07.922876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:07.940039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 197/397] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Jul 2 03:21:07 2022] PUT /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92/groups/c676772b961f496690b42a8bb709ca64/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:07.942500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0cb51-614b-47e6-82a9-1e1f526bd28d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.942812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0cb51-614b-47e6-82a9-1e1f526bd28d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.943054 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0cb51-614b-47e6-82a9-1e1f526bd28d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.947418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 198/398] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:07.951579 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b909ed-69e3-4af5-be20-4a1522c84a03 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.952212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b909ed-69e3-4af5-be20-4a1522c84a03 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.952694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b909ed-69e3-4af5-be20-4a1522c84a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.962405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32bce0-b499-4ad1-bd0d-24dd9ac3eec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:07.963214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32bce0-b499-4ad1-bd0d-24dd9ac3eec4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:07.963848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32bce0-b499-4ad1-bd0d-24dd9ac3eec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:07.963848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32bce0-b499-4ad1-bd0d-24dd9ac3eec4 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:07.964563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32bce0-b499-4ad1-bd0d-24dd9ac3eec4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:07.964869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32bce0-b499-4ad1-bd0d-24dd9ac3eec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:07.966338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32bce0-b499-4ad1-bd0d-24dd9ac3eec4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:07.972592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-18b909ed-69e3-4af5-be20-4a1522c84a03 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:07.974139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-18b909ed-69e3-4af5-be20-4a1522c84a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:07.976327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 201/399] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:21:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:07.984608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32bce0-b499-4ad1-bd0d-24dd9ac3eec4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:08.053866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 199/400] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:08.125038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b909ed-69e3-4af5-be20-4a1522c84a03 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:08.127376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32bce0-b499-4ad1-bd0d-24dd9ac3eec4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:08.164416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a2390-b6b5-4951-b5d2-4c3b29268a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:08.165285 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a2390-b6b5-4951-b5d2-4c3b29268a3a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.165567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a2390-b6b5-4951-b5d2-4c3b29268a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.165878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a2390-b6b5-4951-b5d2-4c3b29268a3a None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.168217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a2390-b6b5-4951-b5d2-4c3b29268a3a None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:08.169023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a2390-b6b5-4951-b5d2-4c3b29268a3a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:08.171687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 202/401] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:07 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:08.178061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a2390-b6b5-4951-b5d2-4c3b29268a3a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:08.217027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:08.254616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b909ed-69e3-4af5-be20-4a1522c84a03 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:08.282667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:08.292959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a2390-b6b5-4951-b5d2-4c3b29268a3a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:08.293742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435cfd4-a297-4fbd-8360-e09014ca5d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:08.294808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435cfd4-a297-4fbd-8360-e09014ca5d5c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.295231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435cfd4-a297-4fbd-8360-e09014ca5d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.295790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435cfd4-a297-4fbd-8360-e09014ca5d5c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.308232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435cfd4-a297-4fbd-8360-e09014ca5d5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2435cfd4-a297-4fbd-8360-e09014ca5d5c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:08.317167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:08.318066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:08.318836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:08.321929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 200/402] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:08.327760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435cfd4-a297-4fbd-8360-e09014ca5d5c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:08.330810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f122900-cdb6-4b5b-ae0f-82e6797c8984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:08.331659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f122900-cdb6-4b5b-ae0f-82e6797c8984 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.331949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f122900-cdb6-4b5b-ae0f-82e6797c8984 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.332223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f122900-cdb6-4b5b-ae0f-82e6797c8984 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.333535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f122900-cdb6-4b5b-ae0f-82e6797c8984 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:08.335054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f122900-cdb6-4b5b-ae0f-82e6797c8984 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:08.337261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 203/403] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:08 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:08.358821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f122900-cdb6-4b5b-ae0f-82e6797c8984 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:08.449251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435cfd4-a297-4fbd-8360-e09014ca5d5c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:08.489373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef703ec-5adb-4765-a4dc-8056cfcb7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:08.490620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef703ec-5adb-4765-a4dc-8056cfcb7481 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.491131 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef703ec-5adb-4765-a4dc-8056cfcb7481 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.491652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef703ec-5adb-4765-a4dc-8056cfcb7481 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.492592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef703ec-5adb-4765-a4dc-8056cfcb7481 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:08.493094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef703ec-5adb-4765-a4dc-8056cfcb7481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:08.494728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef703ec-5adb-4765-a4dc-8056cfcb7481 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:08.505531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 201/404] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:08.512000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef703ec-5adb-4765-a4dc-8056cfcb7481 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:08.595370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:08.599778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f122900-cdb6-4b5b-ae0f-82e6797c8984 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:08.605490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:08.638732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834f81-d386-49d5-af73-d1674fe5f5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:08.639650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834f81-d386-49d5-af73-d1674fe5f5c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.639925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834f81-d386-49d5-af73-d1674fe5f5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.640195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834f81-d386-49d5-af73-d1674fe5f5c2 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0/groups/c676772b961f496690b42a8bb709ca64/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.642207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834f81-d386-49d5-af73-d1674fe5f5c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b61245437cd4d14997cf3df4ea706f0, group_id=c676772b961f496690b42a8bb709ca64, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:08.643943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834f81-d386-49d5-af73-d1674fe5f5c2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:08.660483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 204/405] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Jul 2 03:21:08 2022] PUT /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0/groups/c676772b961f496690b42a8bb709ca64/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:21:08.667481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9d214-eb5f-46b0-a3ea-f260c91c284e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.667481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9d214-eb5f-46b0-a3ea-f260c91c284e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.667481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9d214-eb5f-46b0-a3ea-f260c91c284e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.667816 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 205/406] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:08.670049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb4fd3-072a-4f64-940e-22ad8ff44f50 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.670339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb4fd3-072a-4f64-940e-22ad8ff44f50 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.670607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb4fd3-072a-4f64-940e-22ad8ff44f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.678162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef703ec-5adb-4765-a4dc-8056cfcb7481 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:08.690056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-01fb4fd3-072a-4f64-940e-22ad8ff44f50 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:08.690439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-01fb4fd3-072a-4f64-940e-22ad8ff44f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:08.742635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9dc48e-787c-492e-8184-f6891d613ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:08.743879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9dc48e-787c-492e-8184-f6891d613ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.744501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9dc48e-787c-492e-8184-f6891d613ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.745036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9dc48e-787c-492e-8184-f6891d613ed9 None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.748970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9dc48e-787c-492e-8184-f6891d613ed9 None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:08.751036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9dc48e-787c-492e-8184-f6891d613ed9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:08.753409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 202/407] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:08 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:08.760210 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9dc48e-787c-492e-8184-f6891d613ed9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:08.772171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 206/408] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:08.795233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb4fd3-072a-4f64-940e-22ad8ff44f50 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:08.891637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:08.902129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9dc48e-787c-492e-8184-f6891d613ed9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:08.906155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb4fd3-072a-4f64-940e-22ad8ff44f50 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:08.920708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:08.941219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0b9c8-3566-433c-b732-88ff372854fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:08.942479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0b9c8-3566-433c-b732-88ff372854fd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.942958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0b9c8-3566-433c-b732-88ff372854fd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.943452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0b9c8-3566-433c-b732-88ff372854fd None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.944843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0b9c8-3566-433c-b732-88ff372854fd None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:08.946530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0b9c8-3566-433c-b732-88ff372854fd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:08.947276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1635ae-3e4c-4e8f-a5ef-cfa9c410c4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:08.948301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1635ae-3e4c-4e8f-a5ef-cfa9c410c4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:08.948581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1635ae-3e4c-4e8f-a5ef-cfa9c410c4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:08.948854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1635ae-3e4c-4e8f-a5ef-cfa9c410c4a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:08.949816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 203/409] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:08 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:08.956046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0b9c8-3566-433c-b732-88ff372854fd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:08.961281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1635ae-3e4c-4e8f-a5ef-cfa9c410c4a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-db1635ae-3e4c-4e8f-a5ef-cfa9c410c4a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:08.969378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:08.970295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:08.971061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 207/410] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:08.976969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1635ae-3e4c-4e8f-a5ef-cfa9c410c4a4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:09.071185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0b9c8-3566-433c-b732-88ff372854fd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:09.096888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1635ae-3e4c-4e8f-a5ef-cfa9c410c4a4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:09.120855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ebe8d-f7d9-4620-9147-bdacb727c79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:09.122072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ebe8d-f7d9-4620-9147-bdacb727c79f None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:09.122799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ebe8d-f7d9-4620-9147-bdacb727c79f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:09.123153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ebe8d-f7d9-4620-9147-bdacb727c79f None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0/groups/c676772b961f496690b42a8bb709ca64/roles/62ac4326d6d742c5b6684746953362b1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:09.126046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ebe8d-f7d9-4620-9147-bdacb727c79f None admin] RBAC: Authorizing `identity:create_grant(project_id=4b61245437cd4d14997cf3df4ea706f0, group_id=c676772b961f496690b42a8bb709ca64, role_id=62ac4326d6d742c5b6684746953362b1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:09.127907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ebe8d-f7d9-4620-9147-bdacb727c79f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:09.144551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 204/411] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Jul 2 03:21:08 2022] PUT /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0/groups/c676772b961f496690b42a8bb709ca64/roles/62ac4326d6d742c5b6684746953362b1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:21:09.145669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615397e8-c147-47f0-a6aa-be3409339a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:09.146533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615397e8-c147-47f0-a6aa-be3409339a8a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:09.146983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615397e8-c147-47f0-a6aa-be3409339a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:09.147395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615397e8-c147-47f0-a6aa-be3409339a8a None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:09.148227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615397e8-c147-47f0-a6aa-be3409339a8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:09.148603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615397e8-c147-47f0-a6aa-be3409339a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:09.150279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615397e8-c147-47f0-a6aa-be3409339a8a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:09.152180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed14e6f-791d-497c-bee6-10b45a55c0ab None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:09.152476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed14e6f-791d-497c-bee6-10b45a55c0ab None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:09.152760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed14e6f-791d-497c-bee6-10b45a55c0ab None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:09.155530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 205/412] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:09.165321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f00b4e-68d4-4484-8e2a-e4644e520e42 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:09.165873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f00b4e-68d4-4484-8e2a-e4644e520e42 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:09.166228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f00b4e-68d4-4484-8e2a-e4644e520e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:09.169029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 208/413] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:09.174772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615397e8-c147-47f0-a6aa-be3409339a8a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:09.197325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:09.207937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-09f00b4e-68d4-4484-8e2a-e4644e520e42 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:09.209425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-09f00b4e-68d4-4484-8e2a-e4644e520e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:09.297019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:09.345352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 206/414] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:09.379725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f00b4e-68d4-4484-8e2a-e4644e520e42 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:09.397406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615397e8-c147-47f0-a6aa-be3409339a8a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:09.436401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac02fbd-a308-4526-82c9-dba6aaea6e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:09.437542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac02fbd-a308-4526-82c9-dba6aaea6e00 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:09.437961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac02fbd-a308-4526-82c9-dba6aaea6e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:09.438366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac02fbd-a308-4526-82c9-dba6aaea6e00 None admin] PATH_INFO: `/v3/groups/060862c82dcd43e5a038b1428e761b90` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:09.439712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac02fbd-a308-4526-82c9-dba6aaea6e00 None admin] RBAC: Authorizing `identity:get_group(group_id=060862c82dcd43e5a038b1428e761b90)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:09.441388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac02fbd-a308-4526-82c9-dba6aaea6e00 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:09.443852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 209/415] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:09 2022] GET /identity/v3/groups/060862c82dcd43e5a038b1428e761b90 => generated 229 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:09.451462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac02fbd-a308-4526-82c9-dba6aaea6e00 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:09.507058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f00b4e-68d4-4484-8e2a-e4644e520e42 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:09.540358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:09.550628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220eb27-e382-40ff-96b7-b229f4840bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:09.552643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220eb27-e382-40ff-96b7-b229f4840bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:09.553883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220eb27-e382-40ff-96b7-b229f4840bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:09.554475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220eb27-e382-40ff-96b7-b229f4840bd0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:09.578691 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220eb27-e382-40ff-96b7-b229f4840bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:09.587981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac02fbd-a308-4526-82c9-dba6aaea6e00 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2220eb27-e382-40ff-96b7-b229f4840bd0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:09.593485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:09.594361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:09.595166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:09.600805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 207/416] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:09.607875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220eb27-e382-40ff-96b7-b229f4840bd0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:09.660133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7997d4-8a0e-4823-92ed-0f1093c2ed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:09.662078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7997d4-8a0e-4823-92ed-0f1093c2ed3c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:09.663714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7997d4-8a0e-4823-92ed-0f1093c2ed3c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:09.665323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7997d4-8a0e-4823-92ed-0f1093c2ed3c None admin] PATH_INFO: `/v3/projects/08e65693c4ea4b43b13797fe93e3a44a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:09.666994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7997d4-8a0e-4823-92ed-0f1093c2ed3c None admin] RBAC: Authorizing `identity:get_project(project_id=08e65693c4ea4b43b13797fe93e3a44a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:09.671717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7997d4-8a0e-4823-92ed-0f1093c2ed3c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:09.673633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 210/417] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:09 2022] GET /identity/v3/projects/08e65693c4ea4b43b13797fe93e3a44a => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:09.674978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:09.679369 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7997d4-8a0e-4823-92ed-0f1093c2ed3c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:09.778771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220eb27-e382-40ff-96b7-b229f4840bd0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:09.822967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2c726-e856-49a8-984b-a909790250dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:09.824077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2c726-e856-49a8-984b-a909790250dc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:09.824542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2c726-e856-49a8-984b-a909790250dc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:09.824955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2c726-e856-49a8-984b-a909790250dc None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:09.825975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2c726-e856-49a8-984b-a909790250dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:09.826457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2c726-e856-49a8-984b-a909790250dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:09.828121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2c726-e856-49a8-984b-a909790250dc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:09.839327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7997d4-8a0e-4823-92ed-0f1093c2ed3c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:09.847651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 208/418] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:09.853410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2c726-e856-49a8-984b-a909790250dc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:09.883201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c417a-cd72-4099-92a0-13e7519bab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:09.884397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c417a-cd72-4099-92a0-13e7519bab9a None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:09.885341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c417a-cd72-4099-92a0-13e7519bab9a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:09.885754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c417a-cd72-4099-92a0-13e7519bab9a None admin] PATH_INFO: `/v3/projects/08e65693c4ea4b43b13797fe93e3a44a/groups/060862c82dcd43e5a038b1428e761b90/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:09.892946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c417a-cd72-4099-92a0-13e7519bab9a None admin] RBAC: Authorizing `identity:create_grant(project_id=08e65693c4ea4b43b13797fe93e3a44a, group_id=060862c82dcd43e5a038b1428e761b90, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:09.894725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c417a-cd72-4099-92a0-13e7519bab9a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:09.920614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 211/419] 10.222.0.42 () {58 vars in 1442 bytes} [Sat Jul 2 03:21:09 2022] PUT /identity/v3/projects/08e65693c4ea4b43b13797fe93e3a44a/groups/060862c82dcd43e5a038b1428e761b90/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:10.006318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:10.018652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2c726-e856-49a8-984b-a909790250dc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:10.066304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737e333-bc38-419a-9ff4-bb42003cde4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:10.067384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e333-bc38-419a-9ff4-bb42003cde4a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:10.067925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e333-bc38-419a-9ff4-bb42003cde4a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:10.068568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737e333-bc38-419a-9ff4-bb42003cde4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:10.160045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737e333-bc38-419a-9ff4-bb42003cde4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:10.161530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737e333-bc38-419a-9ff4-bb42003cde4a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:10.265735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:10.271475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c417a-cd72-4099-92a0-13e7519bab9a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:10.271689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 209/420] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:10.290658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:10.325283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050a5af-ce9e-4b91-9d7b-7d36beb20e27 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:10.325907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050a5af-ce9e-4b91-9d7b-7d36beb20e27 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:10.326454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050a5af-ce9e-4b91-9d7b-7d36beb20e27 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:10.328157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 210/421] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:21:10.333668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d5c56-a69c-4305-981a-e7b6d31108a2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:10.333947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d5c56-a69c-4305-981a-e7b6d31108a2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:10.334222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d5c56-a69c-4305-981a-e7b6d31108a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:10.363214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-860d5c56-a69c-4305-981a-e7b6d31108a2 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:10.364362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-860d5c56-a69c-4305-981a-e7b6d31108a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:10.443326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c417a-cd72-4099-92a0-13e7519bab9a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:10.485612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1f870-e2c4-4206-af7a-e54677980aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:10.486592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1f870-e2c4-4206-af7a-e54677980aac None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:10.486867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1f870-e2c4-4206-af7a-e54677980aac None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:10.487137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1f870-e2c4-4206-af7a-e54677980aac None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:10.492393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1f870-e2c4-4206-af7a-e54677980aac None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:10.494006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1f870-e2c4-4206-af7a-e54677980aac None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:10.502351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 212/422] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:10 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:10.504431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1f870-e2c4-4206-af7a-e54677980aac None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:10.509183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 211/423] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:10.532367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d5c56-a69c-4305-981a-e7b6d31108a2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:10.590459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:10.614104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1f870-e2c4-4206-af7a-e54677980aac None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:10.648438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ddf34-1d73-462f-8218-d82bc9af5206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:10.649304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ddf34-1d73-462f-8218-d82bc9af5206 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:10.649587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ddf34-1d73-462f-8218-d82bc9af5206 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:10.649895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ddf34-1d73-462f-8218-d82bc9af5206 None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:10.651176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ddf34-1d73-462f-8218-d82bc9af5206 None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:10.654242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ddf34-1d73-462f-8218-d82bc9af5206 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:10.654375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 213/424] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:10 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:10.660445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ddf34-1d73-462f-8218-d82bc9af5206 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:10.731554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d5c56-a69c-4305-981a-e7b6d31108a2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:10.764118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ddf34-1d73-462f-8218-d82bc9af5206 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:10.787647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:10.798728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7929e8-4a68-409c-b117-1ad9a7cfe58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:10.799442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7929e8-4a68-409c-b117-1ad9a7cfe58f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:10.799730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7929e8-4a68-409c-b117-1ad9a7cfe58f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:10.800010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7929e8-4a68-409c-b117-1ad9a7cfe58f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:10.800841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7929e8-4a68-409c-b117-1ad9a7cfe58f None admin] RBAC: Adding query filter params (group.id=c676772b961f496690b42a8bb709ca64, role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=94c3ce30be2e489e9210838c8132fd92) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:10.801153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7929e8-4a68-409c-b117-1ad9a7cfe58f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:10.802598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7929e8-4a68-409c-b117-1ad9a7cfe58f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:10.809374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 214/425] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:10 2022] GET /identity/v3/role_assignments?group.id=c676772b961f496690b42a8bb709ca64&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=62ac4326d6d742c5b6684746953362b1 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:10.812607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fd652-6fcd-4963-8842-9441f0bb704a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:10.813433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fd652-6fcd-4963-8842-9441f0bb704a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:10.813714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fd652-6fcd-4963-8842-9441f0bb704a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:10.813988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fd652-6fcd-4963-8842-9441f0bb704a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:10.844508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fd652-6fcd-4963-8842-9441f0bb704a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5b1fd652-6fcd-4963-8842-9441f0bb704a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:10.853017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:10.854117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:10.854893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 212/426] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:10.862944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7929e8-4a68-409c-b117-1ad9a7cfe58f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:10.880411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:10.984864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7929e8-4a68-409c-b117-1ad9a7cfe58f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:11.020176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d568254-32ee-45b5-bb85-5329008f3e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:11.020857 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568254-32ee-45b5-bb85-5329008f3e1d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.021141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568254-32ee-45b5-bb85-5329008f3e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.021415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568254-32ee-45b5-bb85-5329008f3e1d None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.022166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d568254-32ee-45b5-bb85-5329008f3e1d None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:11.022472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d568254-32ee-45b5-bb85-5329008f3e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:11.023919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d568254-32ee-45b5-bb85-5329008f3e1d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:11.033348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 215/427] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:21:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:11.039171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fd652-6fcd-4963-8842-9441f0bb704a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:11.167489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fd652-6fcd-4963-8842-9441f0bb704a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:11.201360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e5f07-b856-49b4-aa6d-0c5061b2f5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:11.202337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e5f07-b856-49b4-aa6d-0c5061b2f5da None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.202797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e5f07-b856-49b4-aa6d-0c5061b2f5da None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.203247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e5f07-b856-49b4-aa6d-0c5061b2f5da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.209204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d3ad9-54d0-4033-a961-290c252a554d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.209741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d3ad9-54d0-4033-a961-290c252a554d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.210297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d3ad9-54d0-4033-a961-290c252a554d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.212142 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 216/428] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:11.220580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a7951-bf8f-4ed0-ba6f-f70f9efe5c22 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.221137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a7951-bf8f-4ed0-ba6f-f70f9efe5c22 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.221682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a7951-bf8f-4ed0-ba6f-f70f9efe5c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.243164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-455a7951-bf8f-4ed0-ba6f-f70f9efe5c22 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:11.244732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-455a7951-bf8f-4ed0-ba6f-f70f9efe5c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:11.270530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:11.283256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e5f07-b856-49b4-aa6d-0c5061b2f5da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:11.284969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e5f07-b856-49b4-aa6d-0c5061b2f5da None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:11.316490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:11.331670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 217/429] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:11.352633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a7951-bf8f-4ed0-ba6f-f70f9efe5c22 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:11.412431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 213/430] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:11.420370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e5f07-b856-49b4-aa6d-0c5061b2f5da None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:11.480084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a7951-bf8f-4ed0-ba6f-f70f9efe5c22 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:11.517690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8fdbb-f572-43cc-9081-11f942f019b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:11.518587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8fdbb-f572-43cc-9081-11f942f019b6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.518961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8fdbb-f572-43cc-9081-11f942f019b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.519330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8fdbb-f572-43cc-9081-11f942f019b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.519536 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:11.532265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8fdbb-f572-43cc-9081-11f942f019b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:11.538418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e5f07-b856-49b4-aa6d-0c5061b2f5da None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-05f8fdbb-f572-43cc-9081-11f942f019b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:11.541531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:11.542600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 02 03:21:11.543566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:11.547627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 218/431] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:11.549417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8fdbb-f572-43cc-9081-11f942f019b6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:11.584302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb60e50-2390-482c-9abd-08bb9d8e0c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:11.586145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb60e50-2390-482c-9abd-08bb9d8e0c86 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.586875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb60e50-2390-482c-9abd-08bb9d8e0c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.587371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb60e50-2390-482c-9abd-08bb9d8e0c86 None admin] PATH_INFO: `/v3/projects/94c3ce30be2e489e9210838c8132fd92` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.589294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb60e50-2390-482c-9abd-08bb9d8e0c86 None admin] RBAC: Authorizing `identity:get_project(project_id=94c3ce30be2e489e9210838c8132fd92)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:11.591841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb60e50-2390-482c-9abd-08bb9d8e0c86 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:11.594056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 214/432] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:11 2022] GET /identity/v3/projects/94c3ce30be2e489e9210838c8132fd92 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:11.601777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb60e50-2390-482c-9abd-08bb9d8e0c86 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:11.618624 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:11.663558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8fdbb-f572-43cc-9081-11f942f019b6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:11.699977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2a08f-d2f9-449a-a825-85f9dc76d94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:11.700571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2a08f-d2f9-449a-a825-85f9dc76d94c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.700854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2a08f-d2f9-449a-a825-85f9dc76d94c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.701133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2a08f-d2f9-449a-a825-85f9dc76d94c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.702165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2a08f-d2f9-449a-a825-85f9dc76d94c None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:11.702482 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2a08f-d2f9-449a-a825-85f9dc76d94c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:11.704088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d2a08f-d2f9-449a-a825-85f9dc76d94c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:11.713325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb60e50-2390-482c-9abd-08bb9d8e0c86 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:11.715373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 219/433] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:21:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:11.720936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2a08f-d2f9-449a-a825-85f9dc76d94c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:11.754937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b3455-aefe-4cce-9c8b-7d732ee0ddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:11.755824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b3455-aefe-4cce-9c8b-7d732ee0ddc4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.756112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b3455-aefe-4cce-9c8b-7d732ee0ddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.756673 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b3455-aefe-4cce-9c8b-7d732ee0ddc4 None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.757761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b3455-aefe-4cce-9c8b-7d732ee0ddc4 None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:11.759330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b3455-aefe-4cce-9c8b-7d732ee0ddc4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:11.766095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 215/434] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:11 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:11.767945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b3455-aefe-4cce-9c8b-7d732ee0ddc4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:11.792004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:11.840067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2a08f-d2f9-449a-a825-85f9dc76d94c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:11.862501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:11.874483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b3455-aefe-4cce-9c8b-7d732ee0ddc4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:11.877744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b8a18-be73-4cef-a48e-175c3ae8ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:11.878398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b8a18-be73-4cef-a48e-175c3ae8ec09 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.878677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b8a18-be73-4cef-a48e-175c3ae8ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.878948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b8a18-be73-4cef-a48e-175c3ae8ec09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.910705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85f049-97a7-4ecd-bf2a-3407e074acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:11.911393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f049-97a7-4ecd-bf2a-3407e074acee None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.911679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f049-97a7-4ecd-bf2a-3407e074acee None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.911956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f049-97a7-4ecd-bf2a-3407e074acee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.912789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85f049-97a7-4ecd-bf2a-3407e074acee None admin] RBAC: Adding query filter params (group.id=c676772b961f496690b42a8bb709ca64, role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=94c3ce30be2e489e9210838c8132fd92) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:11.913104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85f049-97a7-4ecd-bf2a-3407e074acee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:11.914550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85f049-97a7-4ecd-bf2a-3407e074acee None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:11.927497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 216/435] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:11 2022] GET /identity/v3/role_assignments?group.id=c676772b961f496690b42a8bb709ca64&scope.project.id=94c3ce30be2e489e9210838c8132fd92&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:11.927497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22148157-76eb-4524-b83d-0d5a9188e5cf None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.927497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22148157-76eb-4524-b83d-0d5a9188e5cf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.927497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22148157-76eb-4524-b83d-0d5a9188e5cf None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.927497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 217/436] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:11.930407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0263bd-86eb-4d77-adc2-ed1cdf2c1665 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:11.930407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0263bd-86eb-4d77-adc2-ed1cdf2c1665 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:11.930407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0263bd-86eb-4d77-adc2-ed1cdf2c1665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:11.953062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fe0263bd-86eb-4d77-adc2-ed1cdf2c1665 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:11.954204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fe0263bd-86eb-4d77-adc2-ed1cdf2c1665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:11.964570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b8a18-be73-4cef-a48e-175c3ae8ec09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:11.966094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b8a18-be73-4cef-a48e-175c3ae8ec09 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:12.055644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 218/437] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:12.081813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0263bd-86eb-4d77-adc2-ed1cdf2c1665 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:12.091344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 220/438] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 02 03:21:12.094618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b8a18-be73-4cef-a48e-175c3ae8ec09 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:12.106184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:12.117440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:12.218627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b8a18-be73-4cef-a48e-175c3ae8ec09 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:12.224456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0263bd-86eb-4d77-adc2-ed1cdf2c1665 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:12.263679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c74397-f2d7-482a-be1c-e53d11946588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:12.264679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c74397-f2d7-482a-be1c-e53d11946588 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.265015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c74397-f2d7-482a-be1c-e53d11946588 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.265350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c74397-f2d7-482a-be1c-e53d11946588 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.266837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c74397-f2d7-482a-be1c-e53d11946588 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:12.268272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c74397-f2d7-482a-be1c-e53d11946588 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:12.271387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 221/439] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:12 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:12.273598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb19a3-85af-4c0e-ac40-84a6621226f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:12.274502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb19a3-85af-4c0e-ac40-84a6621226f0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.274785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb19a3-85af-4c0e-ac40-84a6621226f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.275115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb19a3-85af-4c0e-ac40-84a6621226f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.276438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c74397-f2d7-482a-be1c-e53d11946588 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:12.289136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb19a3-85af-4c0e-ac40-84a6621226f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1ccb19a3-85af-4c0e-ac40-84a6621226f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:12.297859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:12.298934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:12.299827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:12.303918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 219/440] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:12.305124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb19a3-85af-4c0e-ac40-84a6621226f0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:12.391273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:12.409031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:12.412192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c74397-f2d7-482a-be1c-e53d11946588 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:12.440599 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb19a3-85af-4c0e-ac40-84a6621226f0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:12.456399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f786e7-829c-4f6f-9b8a-0a77c5a9f510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:12.457259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f786e7-829c-4f6f-9b8a-0a77c5a9f510 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.457534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f786e7-829c-4f6f-9b8a-0a77c5a9f510 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.457801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f786e7-829c-4f6f-9b8a-0a77c5a9f510 None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.459000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f786e7-829c-4f6f-9b8a-0a77c5a9f510 None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:12.460486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f786e7-829c-4f6f-9b8a-0a77c5a9f510 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:12.462128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 222/441] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:12 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:12.463473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bb9fc-5c00-4026-a834-e5a8d14240e1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.463760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bb9fc-5c00-4026-a834-e5a8d14240e1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.464036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bb9fc-5c00-4026-a834-e5a8d14240e1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.465389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 223/442] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:12.469714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bb9fc-5c00-4026-a834-e5a8d14240e1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:12.477645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53002fad-906a-4d67-89df-be482f079214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:12.478377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002fad-906a-4d67-89df-be482f079214 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.478665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002fad-906a-4d67-89df-be482f079214 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.478943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53002fad-906a-4d67-89df-be482f079214 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.479723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53002fad-906a-4d67-89df-be482f079214 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:12.480058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53002fad-906a-4d67-89df-be482f079214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:12.481434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53002fad-906a-4d67-89df-be482f079214 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:12.490329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 220/443] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:12.491792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f119a-869a-4448-aead-468a674d604e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.492077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f119a-869a-4448-aead-468a674d604e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.492351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f119a-869a-4448-aead-468a674d604e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.512053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-404f119a-869a-4448-aead-468a674d604e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:12.513263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-404f119a-869a-4448-aead-468a674d604e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:12.581792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bb9fc-5c00-4026-a834-e5a8d14240e1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:12.587672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 221/444] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:12.588504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f119a-869a-4448-aead-468a674d604e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:12.634156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f7a07-b731-4bdf-879a-8f37182928d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:12.635029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f7a07-b731-4bdf-879a-8f37182928d8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.635329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f7a07-b731-4bdf-879a-8f37182928d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.635605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f7a07-b731-4bdf-879a-8f37182928d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.636429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f7a07-b731-4bdf-879a-8f37182928d8 None admin] RBAC: Adding query filter params (group.id=c676772b961f496690b42a8bb709ca64, role.id=6aab4b825fea4f4292adc656ae17a709, scope.project.id=4b61245437cd4d14997cf3df4ea706f0) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:12.636740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f7a07-b731-4bdf-879a-8f37182928d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:12.638187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f7a07-b731-4bdf-879a-8f37182928d8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:12.643928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 224/445] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:12 2022] GET /identity/v3/role_assignments?group.id=c676772b961f496690b42a8bb709ca64&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=6aab4b825fea4f4292adc656ae17a709 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:12.644693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f7a07-b731-4bdf-879a-8f37182928d8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:12.655652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:12.678393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:12.694639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f119a-869a-4448-aead-468a674d604e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:12.734778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330c5cc-e337-498a-b7e5-08d243fe9d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:12.735214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330c5cc-e337-498a-b7e5-08d243fe9d44 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.735530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330c5cc-e337-498a-b7e5-08d243fe9d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.735811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330c5cc-e337-498a-b7e5-08d243fe9d44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.765457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f7a07-b731-4bdf-879a-8f37182928d8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:12.805442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60ac80-63ef-4d87-82ad-79b484314362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:12.807031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60ac80-63ef-4d87-82ad-79b484314362 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.807386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60ac80-63ef-4d87-82ad-79b484314362 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.807672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60ac80-63ef-4d87-82ad-79b484314362 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.811230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330c5cc-e337-498a-b7e5-08d243fe9d44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:12.812725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330c5cc-e337-498a-b7e5-08d243fe9d44 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:12.820367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60ac80-63ef-4d87-82ad-79b484314362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-4f60ac80-63ef-4d87-82ad-79b484314362 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:12.828291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:12.829184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:12.829959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 225/446] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:12.836084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60ac80-63ef-4d87-82ad-79b484314362 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:12.888940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:12.927716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 222/447] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:12.929763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:12.933734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330c5cc-e337-498a-b7e5-08d243fe9d44 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:12.954957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60ac80-63ef-4d87-82ad-79b484314362 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:12.997796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30b1a9-0008-4f59-b57f-ca0cb399abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:12.999731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30b1a9-0008-4f59-b57f-ca0cb399abfb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:12.999731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30b1a9-0008-4f59-b57f-ca0cb399abfb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:12.999731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30b1a9-0008-4f59-b57f-ca0cb399abfb None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:12.999731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30b1a9-0008-4f59-b57f-ca0cb399abfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:13.000069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30b1a9-0008-4f59-b57f-ca0cb399abfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:13.003322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30b1a9-0008-4f59-b57f-ca0cb399abfb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:13.010254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 226/448] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:13.015677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30b1a9-0008-4f59-b57f-ca0cb399abfb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:13.047285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330c5cc-e337-498a-b7e5-08d243fe9d44 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:13.081016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f9dbf-5296-435b-acb2-ef7da109f491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:13.081862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f9dbf-5296-435b-acb2-ef7da109f491 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:13.082145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f9dbf-5296-435b-acb2-ef7da109f491 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:13.082429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f9dbf-5296-435b-acb2-ef7da109f491 None admin] PATH_INFO: `/v3/projects/4b61245437cd4d14997cf3df4ea706f0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:13.083666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f9dbf-5296-435b-acb2-ef7da109f491 None admin] RBAC: Authorizing `identity:get_project(project_id=4b61245437cd4d14997cf3df4ea706f0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:13.085142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f9dbf-5296-435b-acb2-ef7da109f491 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:13.087015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 223/449] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:12 2022] GET /identity/v3/projects/4b61245437cd4d14997cf3df4ea706f0 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:13.092274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f9dbf-5296-435b-acb2-ef7da109f491 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:13.111473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30b1a9-0008-4f59-b57f-ca0cb399abfb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:13.129797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:13.146158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09363be-3e2e-4da2-81ab-3ef090f67944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:13.146809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09363be-3e2e-4da2-81ab-3ef090f67944 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:13.147090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09363be-3e2e-4da2-81ab-3ef090f67944 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:13.147396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09363be-3e2e-4da2-81ab-3ef090f67944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:13.159816 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:13.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f9dbf-5296-435b-acb2-ef7da109f491 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:13.215719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09363be-3e2e-4da2-81ab-3ef090f67944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:13.217205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09363be-3e2e-4da2-81ab-3ef090f67944 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:13.230240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d90981-ec66-4b5b-9677-60bd41e499da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:13.231090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d90981-ec66-4b5b-9677-60bd41e499da None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:13.231405 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d90981-ec66-4b5b-9677-60bd41e499da None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:13.231684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d90981-ec66-4b5b-9677-60bd41e499da None admin] PATH_INFO: `/v3/groups/c676772b961f496690b42a8bb709ca64` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:13.232871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d90981-ec66-4b5b-9677-60bd41e499da None admin] RBAC: Authorizing `identity:get_group(group_id=c676772b961f496690b42a8bb709ca64)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:13.234308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d90981-ec66-4b5b-9677-60bd41e499da None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:13.235970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 224/450] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:13 2022] GET /identity/v3/groups/c676772b961f496690b42a8bb709ca64 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:13.241831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d90981-ec66-4b5b-9677-60bd41e499da None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:13.316370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 227/451] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 02 03:21:13.322037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09363be-3e2e-4da2-81ab-3ef090f67944 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:13.353448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d90981-ec66-4b5b-9677-60bd41e499da None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:13.355858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:13.364905 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:13.388230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce0cdc-23d9-4c94-a2a5-4371f736cd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:13.388880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce0cdc-23d9-4c94-a2a5-4371f736cd3c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:13.389165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce0cdc-23d9-4c94-a2a5-4371f736cd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:13.389441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce0cdc-23d9-4c94-a2a5-4371f736cd3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:13.390267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce0cdc-23d9-4c94-a2a5-4371f736cd3c None admin] RBAC: Adding query filter params (group.id=c676772b961f496690b42a8bb709ca64, role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=4b61245437cd4d14997cf3df4ea706f0) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:13.390577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce0cdc-23d9-4c94-a2a5-4371f736cd3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:13.392152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce0cdc-23d9-4c94-a2a5-4371f736cd3c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:13.398155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 225/452] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:13 2022] GET /identity/v3/role_assignments?group.id=c676772b961f496690b42a8bb709ca64&scope.project.id=4b61245437cd4d14997cf3df4ea706f0&role.id=62ac4326d6d742c5b6684746953362b1 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:13.423067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09363be-3e2e-4da2-81ab-3ef090f67944 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:13.462584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ac8fb-77d7-4bac-8f75-15180595d3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:13.463746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac8fb-77d7-4bac-8f75-15180595d3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:13.464040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac8fb-77d7-4bac-8f75-15180595d3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:13.464315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ac8fb-77d7-4bac-8f75-15180595d3e8 None admin] PATH_INFO: `/v3/projects/08e65693c4ea4b43b13797fe93e3a44a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:13.465657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ac8fb-77d7-4bac-8f75-15180595d3e8 None admin] RBAC: Authorizing `identity:get_project(project_id=08e65693c4ea4b43b13797fe93e3a44a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:13.467146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ac8fb-77d7-4bac-8f75-15180595d3e8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:13.468923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 228/453] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:21:13 2022] GET /identity/v3/projects/08e65693c4ea4b43b13797fe93e3a44a => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:13.474715 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce0cdc-23d9-4c94-a2a5-4371f736cd3c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:13.580424 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce0cdc-23d9-4c94-a2a5-4371f736cd3c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:13.615735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b882107-7a23-492d-9904-355bd8c98895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:13.616978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882107-7a23-492d-9904-355bd8c98895 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:13.617387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882107-7a23-492d-9904-355bd8c98895 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:13.617791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b882107-7a23-492d-9904-355bd8c98895 None admin] PATH_INFO: `/v3/groups/060862c82dcd43e5a038b1428e761b90` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:13.619664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b882107-7a23-492d-9904-355bd8c98895 None admin] RBAC: Authorizing `identity:get_group(group_id=060862c82dcd43e5a038b1428e761b90)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:13.621883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b882107-7a23-492d-9904-355bd8c98895 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:13.624374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 226/454] 10.222.0.42 () {56 vars in 1180 bytes} [Sat Jul 2 03:21:13 2022] GET /identity/v3/groups/060862c82dcd43e5a038b1428e761b90 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:13.630519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ac8fb-77d7-4bac-8f75-15180595d3e8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:13.736019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ac8fb-77d7-4bac-8f75-15180595d3e8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:13.747849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:13.770938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8c1ba-3a34-44c3-8e9c-6ad623ffd346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:13.771658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8c1ba-3a34-44c3-8e9c-6ad623ffd346 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:13.771943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8c1ba-3a34-44c3-8e9c-6ad623ffd346 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:13.772219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8c1ba-3a34-44c3-8e9c-6ad623ffd346 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:13.773062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8c1ba-3a34-44c3-8e9c-6ad623ffd346 None admin] RBAC: Adding query filter params (group.id=060862c82dcd43e5a038b1428e761b90, role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=08e65693c4ea4b43b13797fe93e3a44a) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:13.773374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8c1ba-3a34-44c3-8e9c-6ad623ffd346 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:13.774849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8c1ba-3a34-44c3-8e9c-6ad623ffd346 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:13.781074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 229/455] 10.222.0.42 () {56 vars in 1376 bytes} [Sat Jul 2 03:21:13 2022] GET /identity/v3/role_assignments?group.id=060862c82dcd43e5a038b1428e761b90&scope.project.id=08e65693c4ea4b43b13797fe93e3a44a&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:16.210349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d38ac-ae1c-470b-a92b-3c5736d13a35 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.211196 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d38ac-ae1c-470b-a92b-3c5736d13a35 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.211644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d38ac-ae1c-470b-a92b-3c5736d13a35 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.213421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 227/456] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:16.218995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a341002-482f-4cca-bcfc-f1e1a848d9d7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.219782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a341002-482f-4cca-bcfc-f1e1a848d9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.220200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a341002-482f-4cca-bcfc-f1e1a848d9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.243520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1a341002-482f-4cca-bcfc-f1e1a848d9d7 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:16.245003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1a341002-482f-4cca-bcfc-f1e1a848d9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:16.345884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 230/457] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:16.383504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d38ac-ae1c-470b-a92b-3c5736d13a35 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:16.429054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2b6a8-db15-4a1f-a567-b473d0bd5c5a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.429711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2b6a8-db15-4a1f-a567-b473d0bd5c5a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.430197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2b6a8-db15-4a1f-a567-b473d0bd5c5a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.432430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 231/458] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:16.434599 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:16.438001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0f695-0ff9-49d8-88c3-571859a1a78f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.438288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0f695-0ff9-49d8-88c3-571859a1a78f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.438564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0f695-0ff9-49d8-88c3-571859a1a78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.475339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ffe0f695-0ff9-49d8-88c3-571859a1a78f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:16.476535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ffe0f695-0ff9-49d8-88c3-571859a1a78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:16.509529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d38ac-ae1c-470b-a92b-3c5736d13a35 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:16.558634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 232/459] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:16.559324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411a4fd-d3bb-48a6-ac55-93309e5f329b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:16.560160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411a4fd-d3bb-48a6-ac55-93309e5f329b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.560446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411a4fd-d3bb-48a6-ac55-93309e5f329b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.560718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411a4fd-d3bb-48a6-ac55-93309e5f329b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.560911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a130f66-c029-450f-b266-30ca6973386a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.561494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a130f66-c029-450f-b266-30ca6973386a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.561902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a130f66-c029-450f-b266-30ca6973386a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.563729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 233/460] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:16.569326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13eda07-7c01-49d8-8066-56503529d98c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.569764 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13eda07-7c01-49d8-8066-56503529d98c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.570189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13eda07-7c01-49d8-8066-56503529d98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.571150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411a4fd-d3bb-48a6-ac55-93309e5f329b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:16.572938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411a4fd-d3bb-48a6-ac55-93309e5f329b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:16.587630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c13eda07-7c01-49d8-8066-56503529d98c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:16.589006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c13eda07-7c01-49d8-8066-56503529d98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d411a4fd-d3bb-48a6-ac55-93309e5f329b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:16.593676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:16.594697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:16.595765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:16.595765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:16.595765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:16.595765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:16.598069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 228/461] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:16.599014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411a4fd-d3bb-48a6-ac55-93309e5f329b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:16.618539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:16.662743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 234/462] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:16.668646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13eda07-7c01-49d8-8066-56503529d98c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:16.707628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:16.724005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411a4fd-d3bb-48a6-ac55-93309e5f329b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:16.766282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1ea7a-219f-44e9-baac-3b63c1f68749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:16.767109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1ea7a-219f-44e9-baac-3b63c1f68749 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.767595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1ea7a-219f-44e9-baac-3b63c1f68749 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.767997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1ea7a-219f-44e9-baac-3b63c1f68749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.781375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1ea7a-219f-44e9-baac-3b63c1f68749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:16.783770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1ea7a-219f-44e9-baac-3b63c1f68749 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:16.804861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13eda07-7c01-49d8-8066-56503529d98c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-bec1ea7a-219f-44e9-baac-3b63c1f68749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:16.806539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:16.807697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:16.808523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:16.808523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:16.808523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:16.808523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:16.808737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 229/463] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:16.809491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1ea7a-219f-44e9-baac-3b63c1f68749 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:16.858444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc834eb-9bd6-47e1-bdbd-0f0526ea2013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:16.859336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc834eb-9bd6-47e1-bdbd-0f0526ea2013 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.859718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc834eb-9bd6-47e1-bdbd-0f0526ea2013 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.860085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc834eb-9bd6-47e1-bdbd-0f0526ea2013 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.860905 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc834eb-9bd6-47e1-bdbd-0f0526ea2013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:16.861332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc834eb-9bd6-47e1-bdbd-0f0526ea2013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:16.863077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc834eb-9bd6-47e1-bdbd-0f0526ea2013 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:16.897029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 235/464] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:16.898488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc834eb-9bd6-47e1-bdbd-0f0526ea2013 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:16.925702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:16.944976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1ea7a-219f-44e9-baac-3b63c1f68749 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:16.982472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70031579-5d99-468b-a431-3e99bde74dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:16.983632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70031579-5d99-468b-a431-3e99bde74dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:16.984120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70031579-5d99-468b-a431-3e99bde74dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:16.984622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70031579-5d99-468b-a431-3e99bde74dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:16.993242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:16.995734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70031579-5d99-468b-a431-3e99bde74dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:16.997697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70031579-5d99-468b-a431-3e99bde74dc3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:17.010331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc834eb-9bd6-47e1-bdbd-0f0526ea2013 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-70031579-5d99-468b-a431-3e99bde74dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:17.016400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:17.017236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:17.018061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:17.018061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:17.018061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:17.018061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:17.023531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 230/465] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:17.024632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70031579-5d99-468b-a431-3e99bde74dc3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:17.045720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d36b67-fcae-4870-97ef-3d2971958c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:17.046635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d36b67-fcae-4870-97ef-3d2971958c43 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.047051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d36b67-fcae-4870-97ef-3d2971958c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.047483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d36b67-fcae-4870-97ef-3d2971958c43 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.048493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d36b67-fcae-4870-97ef-3d2971958c43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:17.049028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d36b67-fcae-4870-97ef-3d2971958c43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:17.050565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d36b67-fcae-4870-97ef-3d2971958c43 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:17.063670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 236/466] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:17.064706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d36b67-fcae-4870-97ef-3d2971958c43 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:17.153417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70031579-5d99-468b-a431-3e99bde74dc3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:17.184074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:17.189010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de0848-c93b-4c1d-aa83-b4359bc1f356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:17.189901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de0848-c93b-4c1d-aa83-b4359bc1f356 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.190324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de0848-c93b-4c1d-aa83-b4359bc1f356 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.190734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de0848-c93b-4c1d-aa83-b4359bc1f356 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.191620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de0848-c93b-4c1d-aa83-b4359bc1f356 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:17.193226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de0848-c93b-4c1d-aa83-b4359bc1f356 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:17.211360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d36b67-fcae-4870-97ef-3d2971958c43 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:17.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 231/467] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:21:17 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:21:17.218354 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de0848-c93b-4c1d-aa83-b4359bc1f356 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:17.266305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106887b-5558-4044-80eb-1b9a0105a71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:17.267134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106887b-5558-4044-80eb-1b9a0105a71e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.267551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106887b-5558-4044-80eb-1b9a0105a71e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.267927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106887b-5558-4044-80eb-1b9a0105a71e None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.269962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106887b-5558-4044-80eb-1b9a0105a71e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:17.270379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106887b-5558-4044-80eb-1b9a0105a71e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:17.271977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106887b-5558-4044-80eb-1b9a0105a71e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:17.287424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 237/468] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:17.288999 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104ef6a-dd32-4599-9c91-6fed43533fd7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.289487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104ef6a-dd32-4599-9c91-6fed43533fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.289951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104ef6a-dd32-4599-9c91-6fed43533fd7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.291996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 238/469] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:17.294380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104ef6a-dd32-4599-9c91-6fed43533fd7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:17.294714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:17.363490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de0848-c93b-4c1d-aa83-b4359bc1f356 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:17.400359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822bc7f-6fea-4c1c-baef-e7ddf09060c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:17.401044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822bc7f-6fea-4c1c-baef-e7ddf09060c5 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.401323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822bc7f-6fea-4c1c-baef-e7ddf09060c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.401598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822bc7f-6fea-4c1c-baef-e7ddf09060c5 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.402323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822bc7f-6fea-4c1c-baef-e7ddf09060c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:17.403830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822bc7f-6fea-4c1c-baef-e7ddf09060c5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:17.409564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0104ef6a-dd32-4599-9c91-6fed43533fd7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:17.428625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 232/470] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:21:17 2022] POST /identity/v3/users => generated 248 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:21:17.430083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7d795-f9f6-4b6a-b7d8-61e1063b1504 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.430378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7d795-f9f6-4b6a-b7d8-61e1063b1504 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.430653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7d795-f9f6-4b6a-b7d8-61e1063b1504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.450148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-53d7d795-f9f6-4b6a-b7d8-61e1063b1504 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:17.451525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-53d7d795-f9f6-4b6a-b7d8-61e1063b1504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:17.452192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c9b08-f3ec-4a34-b15b-2f19357eaf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:17.453105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c9b08-f3ec-4a34-b15b-2f19357eaf49 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.453605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c9b08-f3ec-4a34-b15b-2f19357eaf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.454291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c9b08-f3ec-4a34-b15b-2f19357eaf49 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.456180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c9b08-f3ec-4a34-b15b-2f19357eaf49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:17.457882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c9b08-f3ec-4a34-b15b-2f19357eaf49 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:17.483433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:17.491963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 239/471] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:21:17 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:21:17.564788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 233/472] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:17.590318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c9b08-f3ec-4a34-b15b-2f19357eaf49 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:17.702829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c9b08-f3ec-4a34-b15b-2f19357eaf49 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:17.724254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b1e32-f61c-4be7-ae7a-07feabbd42e2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.724844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b1e32-f61c-4be7-ae7a-07feabbd42e2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.725312 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b1e32-f61c-4be7-ae7a-07feabbd42e2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.727406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 234/473] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:17.735436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaac05-719f-4d45-ac9e-279335877996 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.736108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaac05-719f-4d45-ac9e-279335877996 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.736647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eaac05-719f-4d45-ac9e-279335877996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.738762 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411e162-7818-41bb-95f7-bc8550a738b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:17.739656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411e162-7818-41bb-95f7-bc8550a738b9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.739936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411e162-7818-41bb-95f7-bc8550a738b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.740200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411e162-7818-41bb-95f7-bc8550a738b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.754833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411e162-7818-41bb-95f7-bc8550a738b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:17.756403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411e162-7818-41bb-95f7-bc8550a738b9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:17.760179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-02eaac05-719f-4d45-ac9e-279335877996 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:17.761094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-02eaac05-719f-4d45-ac9e-279335877996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0411e162-7818-41bb-95f7-bc8550a738b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:17.778701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:17.779827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:17.780695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:17.780695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:17.780695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:17.780695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:17.780695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 240/474] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:17.785936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411e162-7818-41bb-95f7-bc8550a738b9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:17.803092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:17.851949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 235/475] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:17.868911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:17.880351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaac05-719f-4d45-ac9e-279335877996 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:17.930440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411e162-7818-41bb-95f7-bc8550a738b9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:17.973893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20efd0-e769-42b9-a3d4-65733f350080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:17.976140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20efd0-e769-42b9-a3d4-65733f350080 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:17.976647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20efd0-e769-42b9-a3d4-65733f350080 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:17.977325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20efd0-e769-42b9-a3d4-65733f350080 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:17.979919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20efd0-e769-42b9-a3d4-65733f350080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:17.983066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20efd0-e769-42b9-a3d4-65733f350080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:17.984746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20efd0-e769-42b9-a3d4-65733f350080 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:18.004248 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 241/476] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:18.011537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20efd0-e769-42b9-a3d4-65733f350080 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:18.079214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eaac05-719f-4d45-ac9e-279335877996 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:18.143016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20efd0-e769-42b9-a3d4-65733f350080 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:18.146226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:18.146907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:18.147189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:18.147488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:18.148248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:18.149654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:18.163779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.sql.core [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 02 03:21:18.163779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 02 03:21:18.163779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [parameters: {'id': 'ae1efa0b5b1548959b077e085c514170', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 02 03:21:18.163779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80354) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 02 03:21:18.166021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() Jul 02 03:21:18.196629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return ref.to_dict() Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 02 03:21:18.197604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application yield resource Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application next(self.gen) Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.rollback() Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.force_reraise() Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise self.value Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application session.commit() Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 02 03:21:18.198498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._prepare_impl() Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.session.flush() Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._flush(objects) Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application compat.raise_( Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application flush_context.execute() Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rec.execute(self) Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 02 03:21:18.199412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 02 03:21:18.200989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result = self._query(query) Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application conn.query(q) Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application result.read() Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 02 03:21:18.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [parameters: {'id': 'ae1efa0b5b1548959b077e085c514170', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:18.202555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 02 03:21:18.204151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:18.206224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 236/477] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Jul 2 03:21:17 2022] POST /identity/v3/roles => generated 144 bytes in 319 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:18.206366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6844e-c499-4ec6-9cdc-a206db4d42db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:18.206366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6844e-c499-4ec6-9cdc-a206db4d42db None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:18.206366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6844e-c499-4ec6-9cdc-a206db4d42db None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:18.206366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6844e-c499-4ec6-9cdc-a206db4d42db None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:18.206902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6844e-c499-4ec6-9cdc-a206db4d42db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:18.206902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6844e-c499-4ec6-9cdc-a206db4d42db None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:18.207156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:18.244183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 242/478] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:21:18 2022] POST /identity/v3/users => generated 249 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:21:18.258183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:18.347094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4506d2-0bf5-4979-8762-13ecfdd41e66 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:18.401061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5541ce-7adb-4956-9b2a-3037487b6dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:18.401962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5541ce-7adb-4956-9b2a-3037487b6dcb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:18.402289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5541ce-7adb-4956-9b2a-3037487b6dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:18.402575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5541ce-7adb-4956-9b2a-3037487b6dcb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:18.420313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5541ce-7adb-4956-9b2a-3037487b6dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ce5541ce-7adb-4956-9b2a-3037487b6dcb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:18.428866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:18.429812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 02 03:21:18.430610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:18.435063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 237/479] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:18.436409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6844e-c499-4ec6-9cdc-a206db4d42db None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:18.550590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6844e-c499-4ec6-9cdc-a206db4d42db None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:18.586113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810f1ab-0b10-4e68-9a75-a7aa2819d0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:18.586796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f1ab-0b10-4e68-9a75-a7aa2819d0bf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:18.587075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f1ab-0b10-4e68-9a75-a7aa2819d0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:18.587370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f1ab-0b10-4e68-9a75-a7aa2819d0bf None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:18.588794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810f1ab-0b10-4e68-9a75-a7aa2819d0bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:18.589106 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810f1ab-0b10-4e68-9a75-a7aa2819d0bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:18.590505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810f1ab-0b10-4e68-9a75-a7aa2819d0bf None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:18.599506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 243/480] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:19.879179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990afc47-4227-4e26-b8cd-171d84817b0a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:19.879729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990afc47-4227-4e26-b8cd-171d84817b0a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:19.879729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990afc47-4227-4e26-b8cd-171d84817b0a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:19.881387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 238/481] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:19.892980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8e688-1447-47c3-af42-61e0cf9b8101 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:19.893463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8e688-1447-47c3-af42-61e0cf9b8101 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:19.893570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba8e688-1447-47c3-af42-61e0cf9b8101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:19.918157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1ba8e688-1447-47c3-af42-61e0cf9b8101 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:19.919389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1ba8e688-1447-47c3-af42-61e0cf9b8101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:19.928505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118027ef-466e-44d4-bc73-5072117e9e33 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:19.928794 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118027ef-466e-44d4-bc73-5072117e9e33 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:19.929078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118027ef-466e-44d4-bc73-5072117e9e33 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:19.930696 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 239/482] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:19.935894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3e67a-cf5b-40cf-a682-04ac20388af5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:19.936181 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3e67a-cf5b-40cf-a682-04ac20388af5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:19.936460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3e67a-cf5b-40cf-a682-04ac20388af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:19.945120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:19.960512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-07a3e67a-cf5b-40cf-a682-04ac20388af5 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:19.961703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-07a3e67a-cf5b-40cf-a682-04ac20388af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:20.030099 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 240/483] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:20.048891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 244/484] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:20.059802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3e67a-cf5b-40cf-a682-04ac20388af5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.078259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:20.089158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8e688-1447-47c3-af42-61e0cf9b8101 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.159703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3e67a-cf5b-40cf-a682-04ac20388af5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:20.197240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e59dd-af40-40e6-9582-d202a1dce1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:20.198032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e59dd-af40-40e6-9582-d202a1dce1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.198310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e59dd-af40-40e6-9582-d202a1dce1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.198586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e59dd-af40-40e6-9582-d202a1dce1f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.209612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e59dd-af40-40e6-9582-d202a1dce1f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-400e59dd-af40-40e6-9582-d202a1dce1f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:20.217393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:20.218483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:20.219262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 241/485] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:20.220386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af472c55-0b9a-4c13-a507-dcca5cdf40c8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.220694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af472c55-0b9a-4c13-a507-dcca5cdf40c8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.223333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af472c55-0b9a-4c13-a507-dcca5cdf40c8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.223333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 242/486] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:20.226971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af472c55-0b9a-4c13-a507-dcca5cdf40c8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.250445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba8e688-1447-47c3-af42-61e0cf9b8101 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:20.288583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef781e5-ddff-4974-8f3d-499128bea1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:20.289601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef781e5-ddff-4974-8f3d-499128bea1bc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.290017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef781e5-ddff-4974-8f3d-499128bea1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.290413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef781e5-ddff-4974-8f3d-499128bea1bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.303430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef781e5-ddff-4974-8f3d-499128bea1bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:20.311581 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7ef781e5-ddff-4974-8f3d-499128bea1bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:20.313281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:20.314817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 245/487] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:20.317282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916db5-68bd-42de-a52e-1c88153eda43 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.319641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916db5-68bd-42de-a52e-1c88153eda43 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.319641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec916db5-68bd-42de-a52e-1c88153eda43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.339319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ec916db5-68bd-42de-a52e-1c88153eda43 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:20.339319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ec916db5-68bd-42de-a52e-1c88153eda43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:20.339775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af472c55-0b9a-4c13-a507-dcca5cdf40c8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:20.353658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:20.372991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cfc31-bdce-455d-aac0-1c55f79e18c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:20.373657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cfc31-bdce-455d-aac0-1c55f79e18c7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.379315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cfc31-bdce-455d-aac0-1c55f79e18c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.379315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cfc31-bdce-455d-aac0-1c55f79e18c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.379315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cfc31-bdce-455d-aac0-1c55f79e18c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:20.379315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cfc31-bdce-455d-aac0-1c55f79e18c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:20.379315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cfc31-bdce-455d-aac0-1c55f79e18c7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:20.385189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 243/488] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:20.386114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cfc31-bdce-455d-aac0-1c55f79e18c7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.416692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 246/489] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:20.417567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec916db5-68bd-42de-a52e-1c88153eda43 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.508931 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cfc31-bdce-455d-aac0-1c55f79e18c7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:20.524321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec916db5-68bd-42de-a52e-1c88153eda43 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:20.543277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd318d-3530-4ce1-a663-9c73bd4168f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:20.548464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd318d-3530-4ce1-a663-9c73bd4168f0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.548464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd318d-3530-4ce1-a663-9c73bd4168f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.548464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd318d-3530-4ce1-a663-9c73bd4168f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.548464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd318d-3530-4ce1-a663-9c73bd4168f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:20.548464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd318d-3530-4ce1-a663-9c73bd4168f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:20.548464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd318d-3530-4ce1-a663-9c73bd4168f0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:20.557104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 244/490] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:20.558183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd318d-3530-4ce1-a663-9c73bd4168f0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.561255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:20.562881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232e5d8-47a1-426b-88c7-5d14e647095d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:20.563680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232e5d8-47a1-426b-88c7-5d14e647095d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.564062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232e5d8-47a1-426b-88c7-5d14e647095d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.564446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232e5d8-47a1-426b-88c7-5d14e647095d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.577074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232e5d8-47a1-426b-88c7-5d14e647095d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:20.578545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232e5d8-47a1-426b-88c7-5d14e647095d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8232e5d8-47a1-426b-88c7-5d14e647095d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:20.598506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:20.599417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:20.600253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:20.600253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:20.600253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:20.600253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:20.600253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 247/491] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:20.601024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232e5d8-47a1-426b-88c7-5d14e647095d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.606040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:20.664842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd318d-3530-4ce1-a663-9c73bd4168f0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:20.700598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d4e36-8e15-44e5-8cd5-5292cf20f4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:20.701755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d4e36-8e15-44e5-8cd5-5292cf20f4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.702242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d4e36-8e15-44e5-8cd5-5292cf20f4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.702698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d4e36-8e15-44e5-8cd5-5292cf20f4c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.711607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232e5d8-47a1-426b-88c7-5d14e647095d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:20.713965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d4e36-8e15-44e5-8cd5-5292cf20f4c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c29d4e36-8e15-44e5-8cd5-5292cf20f4c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:20.721829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:20.722703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:20.723512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:20.726132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 245/492] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:20.727189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d4e36-8e15-44e5-8cd5-5292cf20f4c3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.746964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f951f-980f-430a-93c9-5831044d888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:20.747795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f951f-980f-430a-93c9-5831044d888c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.748071 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f951f-980f-430a-93c9-5831044d888c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.748342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f951f-980f-430a-93c9-5831044d888c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.760622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f951f-980f-430a-93c9-5831044d888c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:20.762026 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f951f-980f-430a-93c9-5831044d888c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9a8f951f-980f-430a-93c9-5831044d888c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:20.779735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:20.780622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:20.781456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:20.781456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:20.781456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:20.781456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:20.781456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 248/493] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:20.782378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f951f-980f-430a-93c9-5831044d888c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.795716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:20.837769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d4e36-8e15-44e5-8cd5-5292cf20f4c3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:20.842333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:20.875615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8399062-0334-4ffd-9fb5-17feee969b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:20.876274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8399062-0334-4ffd-9fb5-17feee969b62 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.876552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8399062-0334-4ffd-9fb5-17feee969b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.876826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8399062-0334-4ffd-9fb5-17feee969b62 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.877547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8399062-0334-4ffd-9fb5-17feee969b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:20.877855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8399062-0334-4ffd-9fb5-17feee969b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:20.879243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8399062-0334-4ffd-9fb5-17feee969b62 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:20.891257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 246/494] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:20.892446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8399062-0334-4ffd-9fb5-17feee969b62 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:20.898233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f951f-980f-430a-93c9-5831044d888c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:20.934974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d500c-3178-4c9b-bf85-129c1a0f4fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:20.935952 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d500c-3178-4c9b-bf85-129c1a0f4fde None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.936399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d500c-3178-4c9b-bf85-129c1a0f4fde None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.936830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d500c-3178-4c9b-bf85-129c1a0f4fde None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.938000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d500c-3178-4c9b-bf85-129c1a0f4fde None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:20.938608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d500c-3178-4c9b-bf85-129c1a0f4fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:20.940643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d500c-3178-4c9b-bf85-129c1a0f4fde None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:20.958219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 249/495] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:20.958219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd42f0a-0080-4843-b9cf-7558da6abb88 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:20.958219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd42f0a-0080-4843-b9cf-7558da6abb88 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:20.958219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd42f0a-0080-4843-b9cf-7558da6abb88 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:20.958219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 250/496] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:20.958219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd42f0a-0080-4843-b9cf-7558da6abb88 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.003281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8399062-0334-4ffd-9fb5-17feee969b62 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.025890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:21.037741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9331353-bc1f-4c49-bf1e-adc90ebddab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.038407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9331353-bc1f-4c49-bf1e-adc90ebddab3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.038690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9331353-bc1f-4c49-bf1e-adc90ebddab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.038973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9331353-bc1f-4c49-bf1e-adc90ebddab3 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.039733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9331353-bc1f-4c49-bf1e-adc90ebddab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:21.040041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9331353-bc1f-4c49-bf1e-adc90ebddab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.041445 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9331353-bc1f-4c49-bf1e-adc90ebddab3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.052376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 247/497] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:21.053572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533d99a-49a0-4648-9ba0-2cf0ab4f829b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.053857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533d99a-49a0-4648-9ba0-2cf0ab4f829b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.054135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533d99a-49a0-4648-9ba0-2cf0ab4f829b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.068528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd42f0a-0080-4843-b9cf-7558da6abb88 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.071879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3533d99a-49a0-4648-9ba0-2cf0ab4f829b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:21.073148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3533d99a-49a0-4648-9ba0-2cf0ab4f829b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:21.087786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:21.111860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc84412-eb7d-4f48-9e01-79563edd76cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.112667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc84412-eb7d-4f48-9e01-79563edd76cc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.112943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc84412-eb7d-4f48-9e01-79563edd76cc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.113213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc84412-eb7d-4f48-9e01-79563edd76cc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.121480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc84412-eb7d-4f48-9e01-79563edd76cc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.122928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc84412-eb7d-4f48-9e01-79563edd76cc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.150069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 248/498] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:21.150069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533d99a-49a0-4648-9ba0-2cf0ab4f829b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9bc84412-eb7d-4f48-9e01-79563edd76cc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:21.150380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:21.151358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 251/499] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:21.152328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc84412-eb7d-4f48-9e01-79563edd76cc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.262982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc84412-eb7d-4f48-9e01-79563edd76cc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.263873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533d99a-49a0-4648-9ba0-2cf0ab4f829b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.277351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:21.296566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b92edf-0f33-429c-8725-1a27c54456e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.297362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b92edf-0f33-429c-8725-1a27c54456e9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.297652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b92edf-0f33-429c-8725-1a27c54456e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.297925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b92edf-0f33-429c-8725-1a27c54456e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.298044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2ac95-a23e-4e89-8aa6-d45f2387ce5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.298733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2ac95-a23e-4e89-8aa6-d45f2387ce5b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.299014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2ac95-a23e-4e89-8aa6-d45f2387ce5b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.299376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2ac95-a23e-4e89-8aa6-d45f2387ce5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.305211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b92edf-0f33-429c-8725-1a27c54456e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.307027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b92edf-0f33-429c-8725-1a27c54456e9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.309685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2ac95-a23e-4e89-8aa6-d45f2387ce5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.311122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d2ac95-a23e-4e89-8aa6-d45f2387ce5b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-14b92edf-0f33-429c-8725-1a27c54456e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:21.320693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:21.321553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:21.322530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 252/500] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:21.323720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b92edf-0f33-429c-8725-1a27c54456e9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-22d2ac95-a23e-4e89-8aa6-d45f2387ce5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:21.328522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:21.329490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:21.330370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:21.330370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:21.330370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:21.330370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:21.330370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 249/501] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:21.331076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2ac95-a23e-4e89-8aa6-d45f2387ce5b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.359788 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:21.426184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2ac95-a23e-4e89-8aa6-d45f2387ce5b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.431387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b92edf-0f33-429c-8725-1a27c54456e9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.460960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4940cd2-4894-4f60-8607-fb5f36fa6bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.461758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4940cd2-4894-4f60-8607-fb5f36fa6bfd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.462051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4940cd2-4894-4f60-8607-fb5f36fa6bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.462345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4940cd2-4894-4f60-8607-fb5f36fa6bfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.464386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9021d-5fd3-452a-ab9e-6a932a96dc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.465063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9021d-5fd3-452a-ab9e-6a932a96dc38 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.465335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9021d-5fd3-452a-ab9e-6a932a96dc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.465606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9021d-5fd3-452a-ab9e-6a932a96dc38 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.466438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9021d-5fd3-452a-ab9e-6a932a96dc38 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:21.466748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9021d-5fd3-452a-ab9e-6a932a96dc38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.468226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9021d-5fd3-452a-ab9e-6a932a96dc38 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.472422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4940cd2-4894-4f60-8607-fb5f36fa6bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b4940cd2-4894-4f60-8607-fb5f36fa6bfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:21.480135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:21.481117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:21.482050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:21.486340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 250/502] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:21.487648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edee6b-e1f1-4096-9d42-94d73f3411a3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.487935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edee6b-e1f1-4096-9d42-94d73f3411a3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.488209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edee6b-e1f1-4096-9d42-94d73f3411a3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.489644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 251/503] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:21.490323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edee6b-e1f1-4096-9d42-94d73f3411a3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.507835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 253/504] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:21.508712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9021d-5fd3-452a-ab9e-6a932a96dc38 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.518141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:21.596905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edee6b-e1f1-4096-9d42-94d73f3411a3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.605753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:21.613328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9021d-5fd3-452a-ab9e-6a932a96dc38 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.630679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a2e17-57be-4601-b1b9-5f9318956571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.631422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a2e17-57be-4601-b1b9-5f9318956571 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.631698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a2e17-57be-4601-b1b9-5f9318956571 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.631971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a2e17-57be-4601-b1b9-5f9318956571 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.632802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a2e17-57be-4601-b1b9-5f9318956571 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:21.633107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a2e17-57be-4601-b1b9-5f9318956571 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.634601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a2e17-57be-4601-b1b9-5f9318956571 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.646895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114afa6-8ef4-4a20-817a-f9b5aa5481b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.647582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114afa6-8ef4-4a20-817a-f9b5aa5481b5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.647860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114afa6-8ef4-4a20-817a-f9b5aa5481b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.648125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114afa6-8ef4-4a20-817a-f9b5aa5481b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.648821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114afa6-8ef4-4a20-817a-f9b5aa5481b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:21.649123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114afa6-8ef4-4a20-817a-f9b5aa5481b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.650587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114afa6-8ef4-4a20-817a-f9b5aa5481b5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.661748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 254/505] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:21.662546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114afa6-8ef4-4a20-817a-f9b5aa5481b5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.670737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 252/506] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:21.672018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa007594-b6ec-47d0-81a0-dea3bdc25ff3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.672339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa007594-b6ec-47d0-81a0-dea3bdc25ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.672613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa007594-b6ec-47d0-81a0-dea3bdc25ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.689588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-aa007594-b6ec-47d0-81a0-dea3bdc25ff3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:21.690765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-aa007594-b6ec-47d0-81a0-dea3bdc25ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:21.756304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 253/507] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:21.757132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa007594-b6ec-47d0-81a0-dea3bdc25ff3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.763732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114afa6-8ef4-4a20-817a-f9b5aa5481b5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.784092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:21.799276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6683df-2ec3-4630-a309-8b8080ed3b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.799979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6683df-2ec3-4630-a309-8b8080ed3b6c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.800261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6683df-2ec3-4630-a309-8b8080ed3b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.800538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6683df-2ec3-4630-a309-8b8080ed3b6c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.801281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6683df-2ec3-4630-a309-8b8080ed3b6c None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:21.801583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6683df-2ec3-4630-a309-8b8080ed3b6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.802951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6683df-2ec3-4630-a309-8b8080ed3b6c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.811236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 255/508] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:21.812051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6683df-2ec3-4630-a309-8b8080ed3b6c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.829504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:21.872558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa007594-b6ec-47d0-81a0-dea3bdc25ff3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.907544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf47de0-5b91-4a46-9c99-73a2d4b236d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.908158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47de0-5b91-4a46-9c99-73a2d4b236d0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.908453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47de0-5b91-4a46-9c99-73a2d4b236d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.908704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47de0-5b91-4a46-9c99-73a2d4b236d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.908823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6683df-2ec3-4630-a309-8b8080ed3b6c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:21.942435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156482a-8ea4-408c-86c3-a64f79b59c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:21.943216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156482a-8ea4-408c-86c3-a64f79b59c0a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:21.943520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156482a-8ea4-408c-86c3-a64f79b59c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:21.943796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156482a-8ea4-408c-86c3-a64f79b59c0a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:21.951498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156482a-8ea4-408c-86c3-a64f79b59c0a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.953042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156482a-8ea4-408c-86c3-a64f79b59c0a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c156482a-8ea4-408c-86c3-a64f79b59c0a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:21.967493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:21.968366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:21.969241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 256/509] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:21.969786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156482a-8ea4-408c-86c3-a64f79b59c0a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:21.982055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf47de0-5b91-4a46-9c99-73a2d4b236d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:21.983629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf47de0-5b91-4a46-9c99-73a2d4b236d0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:21.998097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.047414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.069650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156482a-8ea4-408c-86c3-a64f79b59c0a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.084730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 254/510] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:22.085580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf47de0-5b91-4a46-9c99-73a2d4b236d0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.111760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8055bc-e5e7-401f-836b-a1e12e79abb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.112420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8055bc-e5e7-401f-836b-a1e12e79abb8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.112700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8055bc-e5e7-401f-836b-a1e12e79abb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.112970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8055bc-e5e7-401f-836b-a1e12e79abb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.182965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8055bc-e5e7-401f-836b-a1e12e79abb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.184472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8055bc-e5e7-401f-836b-a1e12e79abb8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:22.187709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf47de0-5b91-4a46-9c99-73a2d4b236d0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.201879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.223036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b9597-7083-43ee-84b6-d10e632f226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.223957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b9597-7083-43ee-84b6-d10e632f226e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.224238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b9597-7083-43ee-84b6-d10e632f226e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.224515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b9597-7083-43ee-84b6-d10e632f226e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.236926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b9597-7083-43ee-84b6-d10e632f226e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.238386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b9597-7083-43ee-84b6-d10e632f226e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-146b9597-7083-43ee-84b6-d10e632f226e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:22.255731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:22.256575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:22.257395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:22.257395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:22.257395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:22.257395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:22.257395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 255/511] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:22.257663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b9597-7083-43ee-84b6-d10e632f226e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.264659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.289043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 257/512] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:22.289868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8055bc-e5e7-401f-836b-a1e12e79abb8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.361552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b9597-7083-43ee-84b6-d10e632f226e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.385682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8055bc-e5e7-401f-836b-a1e12e79abb8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.395936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dc08a-a3d1-443f-b421-b241c2780979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.396728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dc08a-a3d1-443f-b421-b241c2780979 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.397009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dc08a-a3d1-443f-b421-b241c2780979 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.397290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dc08a-a3d1-443f-b421-b241c2780979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.407882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485dc08a-a3d1-443f-b421-b241c2780979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.409345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485dc08a-a3d1-443f-b421-b241c2780979 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:22.417701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410577c0-ad02-408e-9125-b30ff9a15b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.418361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410577c0-ad02-408e-9125-b30ff9a15b13 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.418637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410577c0-ad02-408e-9125-b30ff9a15b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.418910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410577c0-ad02-408e-9125-b30ff9a15b13 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.419764 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410577c0-ad02-408e-9125-b30ff9a15b13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:22.420066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410577c0-ad02-408e-9125-b30ff9a15b13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.421496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410577c0-ad02-408e-9125-b30ff9a15b13 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-485dc08a-a3d1-443f-b421-b241c2780979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:22.426093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:22.427071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:22.433385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:22.433385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:22.433385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:22.433385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:22.433385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 256/513] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:22.433385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dc08a-a3d1-443f-b421-b241c2780979 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.439227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 258/514] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Jul 2 03:21:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:22.440061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410577c0-ad02-408e-9125-b30ff9a15b13 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.447043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.476155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.528969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dc08a-a3d1-443f-b421-b241c2780979 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.540272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410577c0-ad02-408e-9125-b30ff9a15b13 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.562712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd8c8c-e74f-4e82-8e74-75d5754afc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.563527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd8c8c-e74f-4e82-8e74-75d5754afc8e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.563795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd8c8c-e74f-4e82-8e74-75d5754afc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.564065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd8c8c-e74f-4e82-8e74-75d5754afc8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.574434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd8c8c-e74f-4e82-8e74-75d5754afc8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.574574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da633b-1fd6-42fc-9de6-09253505f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.575196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da633b-1fd6-42fc-9de6-09253505f04e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.575503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da633b-1fd6-42fc-9de6-09253505f04e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.575776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da633b-1fd6-42fc-9de6-09253505f04e None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.575910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd8c8c-e74f-4e82-8e74-75d5754afc8e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:22.576489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da633b-1fd6-42fc-9de6-09253505f04e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:22.576807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da633b-1fd6-42fc-9de6-09253505f04e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.578168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da633b-1fd6-42fc-9de6-09253505f04e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:22.589735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 259/515] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:22.590518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da633b-1fd6-42fc-9de6-09253505f04e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-36fd8c8c-e74f-4e82-8e74-75d5754afc8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:22.593158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:22.593983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:22.594817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:22.594817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:22.594817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:22.594817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:22.594817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 257/516] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:22.595971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd8c8c-e74f-4e82-8e74-75d5754afc8e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.662636 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.688615 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.695841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da633b-1fd6-42fc-9de6-09253505f04e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.697595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd8c8c-e74f-4e82-8e74-75d5754afc8e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.728434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85f639-3eed-4b8a-aab8-ad9b0262cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.729231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85f639-3eed-4b8a-aab8-ad9b0262cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.729510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85f639-3eed-4b8a-aab8-ad9b0262cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.729781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85f639-3eed-4b8a-aab8-ad9b0262cdd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.731911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393fb07-1d6c-44be-a5a4-ed620aad4ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.732571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393fb07-1d6c-44be-a5a4-ed620aad4ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.732849 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393fb07-1d6c-44be-a5a4-ed620aad4ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.733119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393fb07-1d6c-44be-a5a4-ed620aad4ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.733834 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393fb07-1d6c-44be-a5a4-ed620aad4ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:22.734137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393fb07-1d6c-44be-a5a4-ed620aad4ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.735561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393fb07-1d6c-44be-a5a4-ed620aad4ce3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:22.740431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85f639-3eed-4b8a-aab8-ad9b0262cdd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.741872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85f639-3eed-4b8a-aab8-ad9b0262cdd4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:22.746941 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 258/517] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:22.747759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393fb07-1d6c-44be-a5a4-ed620aad4ce3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9b85f639-3eed-4b8a-aab8-ad9b0262cdd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:22.758887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:22.760013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:22.760822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:22.760822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:22.760822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:22.760822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:22.760822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 260/518] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:22.761084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85f639-3eed-4b8a-aab8-ad9b0262cdd4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.853943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393fb07-1d6c-44be-a5a4-ed620aad4ce3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.862824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85f639-3eed-4b8a-aab8-ad9b0262cdd4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:22.880939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.889851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3a1d8-409c-4edd-a03d-1daa46f8efea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.890497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3a1d8-409c-4edd-a03d-1daa46f8efea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.890777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3a1d8-409c-4edd-a03d-1daa46f8efea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.891052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3a1d8-409c-4edd-a03d-1daa46f8efea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.895662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fbe6c-6bed-4ff0-a6ef-31d64a3c0954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:22.896324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fbe6c-6bed-4ff0-a6ef-31d64a3c0954 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:22.896608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fbe6c-6bed-4ff0-a6ef-31d64a3c0954 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:22.896880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fbe6c-6bed-4ff0-a6ef-31d64a3c0954 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:22.897580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fbe6c-6bed-4ff0-a6ef-31d64a3c0954 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.899048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fbe6c-6bed-4ff0-a6ef-31d64a3c0954 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:22.919856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 261/519] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:21:22 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:21:22.920609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fbe6c-6bed-4ff0-a6ef-31d64a3c0954 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:22.923460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:22.967920 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3a1d8-409c-4edd-a03d-1daa46f8efea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:22.970064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3a1d8-409c-4edd-a03d-1daa46f8efea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.035085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fbe6c-6bed-4ff0-a6ef-31d64a3c0954 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.080604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32848d-3358-40c9-8458-7ada16e14b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.080953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32848d-3358-40c9-8458-7ada16e14b30 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.081132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32848d-3358-40c9-8458-7ada16e14b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.085762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 259/520] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:23.085894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32848d-3358-40c9-8458-7ada16e14b30 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.085894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32848d-3358-40c9-8458-7ada16e14b30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:23.086023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3a1d8-409c-4edd-a03d-1daa46f8efea None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.086089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32848d-3358-40c9-8458-7ada16e14b30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.086089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32848d-3358-40c9-8458-7ada16e14b30 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.097998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 262/521] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:23.098915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32848d-3358-40c9-8458-7ada16e14b30 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.120020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:23.169999 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:23.197694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3a1d8-409c-4edd-a03d-1daa46f8efea None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.209977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32848d-3358-40c9-8458-7ada16e14b30 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.233190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3f46-b237-4498-9983-6da6f32fb3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.233980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3f46-b237-4498-9983-6da6f32fb3bd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.234256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3f46-b237-4498-9983-6da6f32fb3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.234519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3f46-b237-4498-9983-6da6f32fb3bd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.241715 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c3f46-b237-4498-9983-6da6f32fb3bd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.243153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c3f46-b237-4498-9983-6da6f32fb3bd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.246215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddffa6-05aa-47a0-ab30-d55e4993f3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.246881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddffa6-05aa-47a0-ab30-d55e4993f3ed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.247160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddffa6-05aa-47a0-ab30-d55e4993f3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.247466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddffa6-05aa-47a0-ab30-d55e4993f3ed None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.248189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddffa6-05aa-47a0-ab30-d55e4993f3ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:23.248495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddffa6-05aa-47a0-ab30-d55e4993f3ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.249950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ddffa6-05aa-47a0-ab30-d55e4993f3ed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-ca8c3f46-b237-4498-9983-6da6f32fb3bd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:23.259351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:23.260566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 260/522] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:23.266351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3f46-b237-4498-9983-6da6f32fb3bd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.269894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 263/523] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:23.270939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddffa6-05aa-47a0-ab30-d55e4993f3ed None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.352120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:23.365770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3f46-b237-4498-9983-6da6f32fb3bd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.386138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddffa6-05aa-47a0-ab30-d55e4993f3ed None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.401892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751cfcc-1c71-4f15-960f-9c0d1728584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.402689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751cfcc-1c71-4f15-960f-9c0d1728584d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.402969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751cfcc-1c71-4f15-960f-9c0d1728584d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751cfcc-1c71-4f15-960f-9c0d1728584d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.410901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:23.414471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751cfcc-1c71-4f15-960f-9c0d1728584d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.416068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751cfcc-1c71-4f15-960f-9c0d1728584d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.425380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2fbbf-9379-48d9-8cf2-159db7f0032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.426224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2fbbf-9379-48d9-8cf2-159db7f0032b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.426505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2fbbf-9379-48d9-8cf2-159db7f0032b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.426778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2fbbf-9379-48d9-8cf2-159db7f0032b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.434468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2fbbf-9379-48d9-8cf2-159db7f0032b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6751cfcc-1c71-4f15-960f-9c0d1728584d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:23.435552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:23.436544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:23.437381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:23.437381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:23.437381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:23.437381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:23.437609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2fbbf-9379-48d9-8cf2-159db7f0032b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.437679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 261/524] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:23.437679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751cfcc-1c71-4f15-960f-9c0d1728584d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-44b2fbbf-9379-48d9-8cf2-159db7f0032b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:23.455720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:23.456625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:23.457434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:23.457434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:23.457434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:23.457434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:23.457434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:23.457434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:23.457434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:23.457434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:23.457434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 264/525] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:23.458333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2fbbf-9379-48d9-8cf2-159db7f0032b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.551078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751cfcc-1c71-4f15-960f-9c0d1728584d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.566905 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2fbbf-9379-48d9-8cf2-159db7f0032b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.589298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3832fd-b49f-46d1-b482-1b869287d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.589962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3832fd-b49f-46d1-b482-1b869287d624 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.590245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3832fd-b49f-46d1-b482-1b869287d624 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.590514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3832fd-b49f-46d1-b482-1b869287d624 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.591369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3832fd-b49f-46d1-b482-1b869287d624 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:23.591676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3832fd-b49f-46d1-b482-1b869287d624 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.593128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3832fd-b49f-46d1-b482-1b869287d624 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.602462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a074a-9bdf-4b88-8451-24fc7b39f798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.603424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a074a-9bdf-4b88-8451-24fc7b39f798 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.603680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:23.603784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a074a-9bdf-4b88-8451-24fc7b39f798 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.603973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a074a-9bdf-4b88-8451-24fc7b39f798 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.611688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a074a-9bdf-4b88-8451-24fc7b39f798 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.613157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a074a-9bdf-4b88-8451-24fc7b39f798 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.618670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 262/526] 10.222.0.42 () {56 vars in 1139 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:23.618670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3832fd-b49f-46d1-b482-1b869287d624 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c12a074a-9bdf-4b88-8451-24fc7b39f798 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:23.631558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:23.632415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:23.633254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:23.633254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:23.633254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:23.633254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:23.633254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:23.633254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:23.633254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:23.633254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:23.637523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 265/527] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:23.638384 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a074a-9bdf-4b88-8451-24fc7b39f798 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.685396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:23.726765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3832fd-b49f-46d1-b482-1b869287d624 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.743404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a074a-9bdf-4b88-8451-24fc7b39f798 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.762067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50141-776c-4d50-9b45-f422b4030ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.762918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50141-776c-4d50-9b45-f422b4030ada None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.763359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50141-776c-4d50-9b45-f422b4030ada None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.763774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c50141-776c-4d50-9b45-f422b4030ada None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.764721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50141-776c-4d50-9b45-f422b4030ada None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:23.765070 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50141-776c-4d50-9b45-f422b4030ada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.766470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c50141-776c-4d50-9b45-f422b4030ada None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.777844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc398ded-e070-4557-bfbf-98c3c4208d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.778645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 263/528] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:23.778774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc398ded-e070-4557-bfbf-98c3c4208d79 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.778774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc398ded-e070-4557-bfbf-98c3c4208d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.778997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc398ded-e070-4557-bfbf-98c3c4208d79 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.779794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc398ded-e070-4557-bfbf-98c3c4208d79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:23.780099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc398ded-e070-4557-bfbf-98c3c4208d79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.780571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50141-776c-4d50-9b45-f422b4030ada None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.781555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc398ded-e070-4557-bfbf-98c3c4208d79 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.801139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 266/529] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:23.801803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc398ded-e070-4557-bfbf-98c3c4208d79 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:23.842996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:23.886564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c50141-776c-4d50-9b45-f422b4030ada None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.909021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc398ded-e070-4557-bfbf-98c3c4208d79 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:23.910664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:23.922324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3094c5-3eca-4887-9c34-6c5f31ed9247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.923377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3094c5-3eca-4887-9c34-6c5f31ed9247 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.923838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3094c5-3eca-4887-9c34-6c5f31ed9247 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.924374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3094c5-3eca-4887-9c34-6c5f31ed9247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.944002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaed202-d47a-47b4-a8f1-a989888b6f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:23.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed202-d47a-47b4-a8f1-a989888b6f38 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:23.944859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed202-d47a-47b4-a8f1-a989888b6f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:23.945126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaed202-d47a-47b4-a8f1-a989888b6f38 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:23.945903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaed202-d47a-47b4-a8f1-a989888b6f38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:23.946209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaed202-d47a-47b4-a8f1-a989888b6f38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:23.947998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaed202-d47a-47b4-a8f1-a989888b6f38 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:23.959264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 267/530] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:23.960062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaed202-d47a-47b4-a8f1-a989888b6f38 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:24.005311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3094c5-3eca-4887-9c34-6c5f31ed9247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:24.007562 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3094c5-3eca-4887-9c34-6c5f31ed9247 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:24.062880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaed202-d47a-47b4-a8f1-a989888b6f38 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:24.068161 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:24.100250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7d5e4-89a4-4f7d-897a-6d9ed6ed2572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:24.101450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7d5e4-89a4-4f7d-897a-6d9ed6ed2572 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:24.101741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7d5e4-89a4-4f7d-897a-6d9ed6ed2572 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:24.102017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7d5e4-89a4-4f7d-897a-6d9ed6ed2572 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:24.109251 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7d5e4-89a4-4f7d-897a-6d9ed6ed2572 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:24.110701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7d5e4-89a4-4f7d-897a-6d9ed6ed2572 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:24.124682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 264/531] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:24.124682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3094c5-3eca-4887-9c34-6c5f31ed9247 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-45f7d5e4-89a4-4f7d-897a-6d9ed6ed2572 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:24.125795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:24.126612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 268/532] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:24.128635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7d5e4-89a4-4f7d-897a-6d9ed6ed2572 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:24.160269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:24.233074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3094c5-3eca-4887-9c34-6c5f31ed9247 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:24.237879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7d5e4-89a4-4f7d-897a-6d9ed6ed2572 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:24.268704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad141b-f7ce-49a1-a4e4-7b9132f7f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:24.269350 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad141b-f7ce-49a1-a4e4-7b9132f7f6aa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:24.269630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad141b-f7ce-49a1-a4e4-7b9132f7f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:24.269892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad141b-f7ce-49a1-a4e4-7b9132f7f6aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:24.270723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad141b-f7ce-49a1-a4e4-7b9132f7f6aa None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=e02b8a64165541889ca929b8038412bc) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:24.271037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad141b-f7ce-49a1-a4e4-7b9132f7f6aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:24.272518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad141b-f7ce-49a1-a4e4-7b9132f7f6aa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:24.274691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a07d6-e347-4976-917b-ce24df8fae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:24.275579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a07d6-e347-4976-917b-ce24df8fae27 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:24.275859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a07d6-e347-4976-917b-ce24df8fae27 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:24.276133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a07d6-e347-4976-917b-ce24df8fae27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:24.276954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a07d6-e347-4976-917b-ce24df8fae27 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=8b7bc20034974816905978dd365993f8) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:24.277263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a07d6-e347-4976-917b-ce24df8fae27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:24.283602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 265/533] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:24 2022] GET /identity/v3/role_assignments?user.id=e02b8a64165541889ca929b8038412bc&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:24.283777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a07d6-e347-4976-917b-ce24df8fae27 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:24.283865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad141b-f7ce-49a1-a4e4-7b9132f7f6aa None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:24.290961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 269/534] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:24 2022] GET /identity/v3/role_assignments?user.id=8b7bc20034974816905978dd365993f8&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:24.292634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a07d6-e347-4976-917b-ce24df8fae27 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:24.345073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:24.440678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad141b-f7ce-49a1-a4e4-7b9132f7f6aa None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:24.481242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cab05-0309-4d2d-87e8-fe53fd2f47ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:24.482409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cab05-0309-4d2d-87e8-fe53fd2f47ea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:24.482881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cab05-0309-4d2d-87e8-fe53fd2f47ea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:24.483464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cab05-0309-4d2d-87e8-fe53fd2f47ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:24.495501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4cab05-0309-4d2d-87e8-fe53fd2f47ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:24.497379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4cab05-0309-4d2d-87e8-fe53fd2f47ea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:24.516655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4c4cab05-0309-4d2d-87e8-fe53fd2f47ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:24.518758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:24.519857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:24.520757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:24.520757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:24.520757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:24.520757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:24.524519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 266/535] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:24.528986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cab05-0309-4d2d-87e8-fe53fd2f47ea None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:24.541324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a07d6-e347-4976-917b-ce24df8fae27 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:24.586262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca4624-6c73-4629-95e5-44798d507133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:24.587039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca4624-6c73-4629-95e5-44798d507133 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:24.587439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca4624-6c73-4629-95e5-44798d507133 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:24.587954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca4624-6c73-4629-95e5-44798d507133 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:24.588754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca4624-6c73-4629-95e5-44798d507133 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:24.589414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca4624-6c73-4629-95e5-44798d507133 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:24.590886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca4624-6c73-4629-95e5-44798d507133 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:24.608768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 270/536] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:24.615485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca4624-6c73-4629-95e5-44798d507133 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:24.623687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:24.651031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cab05-0309-4d2d-87e8-fe53fd2f47ea None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:24.688841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796fc64-55d5-499a-a11e-e0543e9e63c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:24.689619 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796fc64-55d5-499a-a11e-e0543e9e63c2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:24.689975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796fc64-55d5-499a-a11e-e0543e9e63c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:24.690331 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796fc64-55d5-499a-a11e-e0543e9e63c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:24.691138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796fc64-55d5-499a-a11e-e0543e9e63c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:24.691546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796fc64-55d5-499a-a11e-e0543e9e63c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:24.693059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796fc64-55d5-499a-a11e-e0543e9e63c2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:24.706296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 267/537] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:24.716374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796fc64-55d5-499a-a11e-e0543e9e63c2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:24.799781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca4624-6c73-4629-95e5-44798d507133 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:24.830797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796fc64-55d5-499a-a11e-e0543e9e63c2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:24.845862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af8d95-4114-4a26-b73f-55416e51a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:24.846630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af8d95-4114-4a26-b73f-55416e51a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:24.846984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af8d95-4114-4a26-b73f-55416e51a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:24.847351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af8d95-4114-4a26-b73f-55416e51a0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:24.848316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af8d95-4114-4a26-b73f-55416e51a0b8 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=2afd25850dea40568271538793e861c3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:24.848708 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af8d95-4114-4a26-b73f-55416e51a0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:24.850263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af8d95-4114-4a26-b73f-55416e51a0b8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:24.859083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 271/538] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:24 2022] GET /identity/v3/role_assignments?user.id=2afd25850dea40568271538793e861c3&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:24.893989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffdb4d-eeac-4a9d-982e-282e7a33f877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:24.894910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffdb4d-eeac-4a9d-982e-282e7a33f877 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:24.895262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffdb4d-eeac-4a9d-982e-282e7a33f877 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:24.895633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffdb4d-eeac-4a9d-982e-282e7a33f877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:24.899416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:24.908421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffdb4d-eeac-4a9d-982e-282e7a33f877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:24.910076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffdb4d-eeac-4a9d-982e-282e7a33f877 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0bffdb4d-eeac-4a9d-982e-282e7a33f877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:24.929441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:24.930359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:24.931376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:24.931376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:24.931376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:24.931376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:24.931376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:24.931376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:24.931376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:24.931376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:24.931376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 268/539] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:24.936849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af8d95-4114-4a26-b73f-55416e51a0b8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:24.956225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:25.068512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af8d95-4114-4a26-b73f-55416e51a0b8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:25.098503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a459c1-3ca8-461d-a5f1-1aa5037d283c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:25.099171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a459c1-3ca8-461d-a5f1-1aa5037d283c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:25.099171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a459c1-3ca8-461d-a5f1-1aa5037d283c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:25.100893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 269/540] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:25.103903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fb556-d13e-4321-8f50-b03fb58a5239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:25.104573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fb556-d13e-4321-8f50-b03fb58a5239 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:25.105367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fb556-d13e-4321-8f50-b03fb58a5239 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:25.105647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fb556-d13e-4321-8f50-b03fb58a5239 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:25.106475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fb556-d13e-4321-8f50-b03fb58a5239 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:25.106744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fb556-d13e-4321-8f50-b03fb58a5239 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:25.108247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0fb556-d13e-4321-8f50-b03fb58a5239 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:25.110983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a52cd-0cc1-4b5b-9512-35d2cba54b9f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:25.111346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a52cd-0cc1-4b5b-9512-35d2cba54b9f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:25.111631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a52cd-0cc1-4b5b-9512-35d2cba54b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:25.123178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 272/541] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:25.129520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fb556-d13e-4321-8f50-b03fb58a5239 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:25.137715 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-524a52cd-0cc1-4b5b-9512-35d2cba54b9f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:25.138961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-524a52cd-0cc1-4b5b-9512-35d2cba54b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:25.210125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:25.243839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fb556-d13e-4321-8f50-b03fb58a5239 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:25.287933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 270/542] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:25.289370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d5a06-745f-426f-b6a4-02dfa5299ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:25.290038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d5a06-745f-426f-b6a4-02dfa5299ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:25.290315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d5a06-745f-426f-b6a4-02dfa5299ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:25.290709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d5a06-745f-426f-b6a4-02dfa5299ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:25.297468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d5a06-745f-426f-b6a4-02dfa5299ac1 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=d67be26446324c91b956e9d930e04b4d) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:25.297468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d5a06-745f-426f-b6a4-02dfa5299ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:25.297468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d5a06-745f-426f-b6a4-02dfa5299ac1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:25.306341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 273/543] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:25 2022] GET /identity/v3/role_assignments?user.id=d67be26446324c91b956e9d930e04b4d&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:25.323217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a52cd-0cc1-4b5b-9512-35d2cba54b9f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:25.426157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a52cd-0cc1-4b5b-9512-35d2cba54b9f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:25.451591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:25.466538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9441efa-b960-4832-941f-19268d49b0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:25.467372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9441efa-b960-4832-941f-19268d49b0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:25.467786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9441efa-b960-4832-941f-19268d49b0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:25.468064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9441efa-b960-4832-941f-19268d49b0e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:25.489330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9441efa-b960-4832-941f-19268d49b0e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b9441efa-b960-4832-941f-19268d49b0e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:25.497336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:25.498285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:25.499052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:25.504527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d5a06-745f-426f-b6a4-02dfa5299ac1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:25.504986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 271/544] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:25.647354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d5a06-745f-426f-b6a4-02dfa5299ac1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:25.692919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9d92e-573a-4dca-9c12-88235cdfdcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:25.693595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9d92e-573a-4dca-9c12-88235cdfdcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:25.693879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9d92e-573a-4dca-9c12-88235cdfdcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:25.694153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9d92e-573a-4dca-9c12-88235cdfdcf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:25.694898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9d92e-573a-4dca-9c12-88235cdfdcf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:25.695206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9d92e-573a-4dca-9c12-88235cdfdcf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:25.696636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9d92e-573a-4dca-9c12-88235cdfdcf9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:25.698027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:25.710837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 274/545] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:25.716459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9441efa-b960-4832-941f-19268d49b0e3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:25.871513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9441efa-b960-4832-941f-19268d49b0e3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:25.915049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d190-8d77-41f4-888b-a30e9b2fa37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:25.915851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d190-8d77-41f4-888b-a30e9b2fa37c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:25.916133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d190-8d77-41f4-888b-a30e9b2fa37c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:25.916408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3d190-8d77-41f4-888b-a30e9b2fa37c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:25.928197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3d190-8d77-41f4-888b-a30e9b2fa37c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:25.929670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3d190-8d77-41f4-888b-a30e9b2fa37c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-dec3d190-8d77-41f4-888b-a30e9b2fa37c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:25.947894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:25.948805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:25.950258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:25.950258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:25.950258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:25.950258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:25.950258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 272/546] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:25.956386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9d92e-573a-4dca-9c12-88235cdfdcf9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:26.106134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9d92e-573a-4dca-9c12-88235cdfdcf9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:26.156886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7c8ae-d7a0-4305-bb19-02d486a522a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:26.157559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7c8ae-d7a0-4305-bb19-02d486a522a4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:26.157840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7c8ae-d7a0-4305-bb19-02d486a522a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:26.158115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7c8ae-d7a0-4305-bb19-02d486a522a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:26.158841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7c8ae-d7a0-4305-bb19-02d486a522a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:26.159144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7c8ae-d7a0-4305-bb19-02d486a522a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:26.160547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7c8ae-d7a0-4305-bb19-02d486a522a4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:26.180576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 275/547] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:26.186313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d190-8d77-41f4-888b-a30e9b2fa37c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:26.226876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:26.325297 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3d190-8d77-41f4-888b-a30e9b2fa37c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:26.373092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e8298-455a-417f-9fe4-e92399cced8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:26.374160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e8298-455a-417f-9fe4-e92399cced8c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:26.374582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e8298-455a-417f-9fe4-e92399cced8c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:26.375006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e8298-455a-417f-9fe4-e92399cced8c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:26.388404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e8298-455a-417f-9fe4-e92399cced8c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:26.390048 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e8298-455a-417f-9fe4-e92399cced8c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-024e8298-455a-417f-9fe4-e92399cced8c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:26.406886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:26.408073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:26.409671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:26.411798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 273/548] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:26.417584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7c8ae-d7a0-4305-bb19-02d486a522a4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:26.514712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:26.573400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7c8ae-d7a0-4305-bb19-02d486a522a4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:26.596449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3fbc3-996d-4b38-93a4-9c4a8597a8dd None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:26.597085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3fbc3-996d-4b38-93a4-9c4a8597a8dd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:26.597552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3fbc3-996d-4b38-93a4-9c4a8597a8dd None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:26.599433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 274/549] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:26.605701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e13fda-a4f5-4c3d-8331-0e1846157e68 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:26.606275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e13fda-a4f5-4c3d-8331-0e1846157e68 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:26.606775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e13fda-a4f5-4c3d-8331-0e1846157e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:26.622438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585da8af-dfc2-4654-9134-375002e76924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:26.623128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585da8af-dfc2-4654-9134-375002e76924 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:26.623436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585da8af-dfc2-4654-9134-375002e76924 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:26.623707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585da8af-dfc2-4654-9134-375002e76924 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:26.624552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585da8af-dfc2-4654-9134-375002e76924 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:26.624860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585da8af-dfc2-4654-9134-375002e76924 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:26.625923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-03e13fda-a4f5-4c3d-8331-0e1846157e68 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:26.626401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585da8af-dfc2-4654-9134-375002e76924 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:26.627605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-03e13fda-a4f5-4c3d-8331-0e1846157e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:26.661394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 276/550] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Jul 2 03:21:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:26.665856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585da8af-dfc2-4654-9134-375002e76924 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:26.749825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 275/551] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:26.812429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e13fda-a4f5-4c3d-8331-0e1846157e68 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:26.830981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585da8af-dfc2-4654-9134-375002e76924 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:26.853915 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:26.854909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:26.904657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665660b-3feb-4d03-b58b-69644a60f414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:26.905312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665660b-3feb-4d03-b58b-69644a60f414 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:26.905593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665660b-3feb-4d03-b58b-69644a60f414 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:26.905868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665660b-3feb-4d03-b58b-69644a60f414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:26.961480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e13fda-a4f5-4c3d-8331-0e1846157e68 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:27.026336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247001fb-6477-478b-959c-188e624d2b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:27.028990 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247001fb-6477-478b-959c-188e624d2b96 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.030246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247001fb-6477-478b-959c-188e624d2b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.031433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247001fb-6477-478b-959c-188e624d2b96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.041838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665660b-3feb-4d03-b58b-69644a60f414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:27.044688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665660b-3feb-4d03-b58b-69644a60f414 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:27.057304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247001fb-6477-478b-959c-188e624d2b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-247001fb-6477-478b-959c-188e624d2b96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:27.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:27.068665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:27.069548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:27.078122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 276/552] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:27.078122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f405c8-f058-4423-b351-9afae1291c78 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.078122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f405c8-f058-4423-b351-9afae1291c78 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.078122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f405c8-f058-4423-b351-9afae1291c78 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.078122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 277/553] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:27.082813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f405c8-f058-4423-b351-9afae1291c78 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:27.184156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:27.198574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 277/554] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:27.200562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff011559-7a84-43fa-8cd7-9b4edccfe174 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.200949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff011559-7a84-43fa-8cd7-9b4edccfe174 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.201318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff011559-7a84-43fa-8cd7-9b4edccfe174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.203374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f405c8-f058-4423-b351-9afae1291c78 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:27.223227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ff011559-7a84-43fa-8cd7-9b4edccfe174 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:27.224601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ff011559-7a84-43fa-8cd7-9b4edccfe174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:27.238549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:27.240937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b479708-aa8b-4499-9158-16c37a40d07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:27.241920 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b479708-aa8b-4499-9158-16c37a40d07d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.242373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b479708-aa8b-4499-9158-16c37a40d07d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.242813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b479708-aa8b-4499-9158-16c37a40d07d None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.244117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b479708-aa8b-4499-9158-16c37a40d07d None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:27.244877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b479708-aa8b-4499-9158-16c37a40d07d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:27.246499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b479708-aa8b-4499-9158-16c37a40d07d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:27.260490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 278/555] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:27.261675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b479708-aa8b-4499-9158-16c37a40d07d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:27.315436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 278/556] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:27.316794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff011559-7a84-43fa-8cd7-9b4edccfe174 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:27.411156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b479708-aa8b-4499-9158-16c37a40d07d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:27.438771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff011559-7a84-43fa-8cd7-9b4edccfe174 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:27.457293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c4c7f-2d48-40fa-a260-0aca1bcf2663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:27.458104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c4c7f-2d48-40fa-a260-0aca1bcf2663 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.458767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c4c7f-2d48-40fa-a260-0aca1bcf2663 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.459062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c4c7f-2d48-40fa-a260-0aca1bcf2663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.475032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c4c7f-2d48-40fa-a260-0aca1bcf2663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:27.476568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c4c7f-2d48-40fa-a260-0aca1bcf2663 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:27.477966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ec7bd-d797-4e9d-bd67-774a8b767728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:27.479156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ec7bd-d797-4e9d-bd67-774a8b767728 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.479683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ec7bd-d797-4e9d-bd67-774a8b767728 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.481227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ec7bd-d797-4e9d-bd67-774a8b767728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.487021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:27.494230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ec7bd-d797-4e9d-bd67-774a8b767728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:27.496343 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ec7bd-d797-4e9d-bd67-774a8b767728 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:27.500128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-485c4c7f-2d48-40fa-a260-0aca1bcf2663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:27.500249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:27.501062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:27.501940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:27.501940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:27.501940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:27.501940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:27.501940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 279/557] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:27.502718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c4c7f-2d48-40fa-a260-0aca1bcf2663 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-419ec7bd-d797-4e9d-bd67-774a8b767728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:27.520967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:27.522421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:27.523350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:27.523350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:27.523350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:27.523350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:27.530395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 279/558] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:27.531551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ec7bd-d797-4e9d-bd67-774a8b767728 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:27.623384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c4c7f-2d48-40fa-a260-0aca1bcf2663 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:27.638256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ec7bd-d797-4e9d-bd67-774a8b767728 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:27.656276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0917f-9a0f-48c5-801a-3bccb0124b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:27.657042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0917f-9a0f-48c5-801a-3bccb0124b41 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.657324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0917f-9a0f-48c5-801a-3bccb0124b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.657598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0917f-9a0f-48c5-801a-3bccb0124b41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.668708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e0917f-9a0f-48c5-801a-3bccb0124b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:27.673896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069ae99-4f57-471a-91db-27f9aaef47e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:27.674571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069ae99-4f57-471a-91db-27f9aaef47e6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.674836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069ae99-4f57-471a-91db-27f9aaef47e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.675112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069ae99-4f57-471a-91db-27f9aaef47e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.675868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069ae99-4f57-471a-91db-27f9aaef47e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:27.676178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069ae99-4f57-471a-91db-27f9aaef47e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-70e0917f-9a0f-48c5-801a-3bccb0124b41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:27.676269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:27.677280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:27.684329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:27.685521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069ae99-4f57-471a-91db-27f9aaef47e6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:27.685645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 280/559] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:27.685645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0917f-9a0f-48c5-801a-3bccb0124b41 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:27.697644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 280/560] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:27.699030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdcb99-f6a0-4928-abea-ad3e1900d1bc None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.699269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdcb99-f6a0-4928-abea-ad3e1900d1bc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.699641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdcb99-f6a0-4928-abea-ad3e1900d1bc None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.701087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 281/561] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:27.701957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd9288-222e-476b-96b6-98d0491a9eed None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.702236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd9288-222e-476b-96b6-98d0491a9eed None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.702532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd9288-222e-476b-96b6-98d0491a9eed None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.707399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 282/562] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:27.707399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd9288-222e-476b-96b6-98d0491a9eed None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:27.755539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:27.771012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:27.796209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0917f-9a0f-48c5-801a-3bccb0124b41 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:27.816811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd9288-222e-476b-96b6-98d0491a9eed None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:27.830587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ffe1-e664-4f36-9311-7412c7914540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:27.831274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ffe1-e664-4f36-9311-7412c7914540 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.831576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ffe1-e664-4f36-9311-7412c7914540 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.831854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0ffe1-e664-4f36-9311-7412c7914540 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.832570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ffe1-e664-4f36-9311-7412c7914540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:27.832864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ffe1-e664-4f36-9311-7412c7914540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:27.834259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0ffe1-e664-4f36-9311-7412c7914540 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:27.846475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 281/563] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:27.847255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ffe1-e664-4f36-9311-7412c7914540 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:27.850341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b238b-ebc1-4353-8f2d-ea95775ebbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:27.851017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b238b-ebc1-4353-8f2d-ea95775ebbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.851696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b238b-ebc1-4353-8f2d-ea95775ebbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.851696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b238b-ebc1-4353-8f2d-ea95775ebbb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.852379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b238b-ebc1-4353-8f2d-ea95775ebbb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:27.852815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b238b-ebc1-4353-8f2d-ea95775ebbb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:27.854177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b238b-ebc1-4353-8f2d-ea95775ebbb6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:27.863669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 283/564] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:27.865197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d26901-f30a-4b0f-893f-77d3f3ac6565 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.865486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d26901-f30a-4b0f-893f-77d3f3ac6565 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.865783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d26901-f30a-4b0f-893f-77d3f3ac6565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.882341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d3d26901-f30a-4b0f-893f-77d3f3ac6565 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:27.883557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d3d26901-f30a-4b0f-893f-77d3f3ac6565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:27.950461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 284/565] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:27.952029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badd8bd-1284-40d1-b91f-5a60199e19bd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.952294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badd8bd-1284-40d1-b91f-5a60199e19bd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.952570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badd8bd-1284-40d1-b91f-5a60199e19bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.954538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0ffe1-e664-4f36-9311-7412c7914540 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:27.968649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0badd8bd-1284-40d1-b91f-5a60199e19bd None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:27.969824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0badd8bd-1284-40d1-b91f-5a60199e19bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:27.974949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:27.989546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d4774-5031-4918-8213-d1b2e5f1570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:27.990337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d4774-5031-4918-8213-d1b2e5f1570d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:27.990615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d4774-5031-4918-8213-d1b2e5f1570d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:27.990883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d4774-5031-4918-8213-d1b2e5f1570d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:27.997644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d4774-5031-4918-8213-d1b2e5f1570d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:27.999085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d4774-5031-4918-8213-d1b2e5f1570d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-eb5d4774-5031-4918-8213-d1b2e5f1570d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:28.012911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:28.013905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:28.014802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:28.014802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:28.014802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:28.014802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:28.014802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:28.014802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:28.014802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:28.014802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:28.015223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:28.015325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 282/566] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:28.015325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d4774-5031-4918-8213-d1b2e5f1570d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.042562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 285/567] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:28.043977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badd8bd-1284-40d1-b91f-5a60199e19bd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.131988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d4774-5031-4918-8213-d1b2e5f1570d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.151578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badd8bd-1284-40d1-b91f-5a60199e19bd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.166508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ca5a7-d74e-46d7-9966-ddee9a4cb032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.167397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ca5a7-d74e-46d7-9966-ddee9a4cb032 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.167684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ca5a7-d74e-46d7-9966-ddee9a4cb032 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.167963 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ca5a7-d74e-46d7-9966-ddee9a4cb032 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.175615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ca5a7-d74e-46d7-9966-ddee9a4cb032 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:28.177102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ca5a7-d74e-46d7-9966-ddee9a4cb032 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.184429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159b30d-e597-4e2d-9b84-c155ccf5a825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.185222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159b30d-e597-4e2d-9b84-c155ccf5a825 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.185501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159b30d-e597-4e2d-9b84-c155ccf5a825 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.185775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f159b30d-e597-4e2d-9b84-c155ccf5a825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f73ca5a7-d74e-46d7-9966-ddee9a4cb032 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:28.191339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:28.192280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:28.193209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 283/568] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:28.193807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ca5a7-d74e-46d7-9966-ddee9a4cb032 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.197733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159b30d-e597-4e2d-9b84-c155ccf5a825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:28.199166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f159b30d-e597-4e2d-9b84-c155ccf5a825 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f159b30d-e597-4e2d-9b84-c155ccf5a825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:28.217602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:28.218480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:28.219548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:28.219548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:28.219548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:28.219548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:28.219548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 286/569] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:28.219548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159b30d-e597-4e2d-9b84-c155ccf5a825 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.239776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:28.249817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:28.299025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ca5a7-d74e-46d7-9966-ddee9a4cb032 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.317750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f159b30d-e597-4e2d-9b84-c155ccf5a825 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.333873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e909c47-fc40-4d03-b0be-ef6e8f2df054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.334667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e909c47-fc40-4d03-b0be-ef6e8f2df054 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.334947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e909c47-fc40-4d03-b0be-ef6e8f2df054 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.335218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e909c47-fc40-4d03-b0be-ef6e8f2df054 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.345254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e909c47-fc40-4d03-b0be-ef6e8f2df054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:28.350547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b898d-e2af-432b-9e2d-f4c12d3a16a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.351194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b898d-e2af-432b-9e2d-f4c12d3a16a7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.351539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b898d-e2af-432b-9e2d-f4c12d3a16a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.351813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b898d-e2af-432b-9e2d-f4c12d3a16a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.352640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b898d-e2af-432b-9e2d-f4c12d3a16a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3e909c47-fc40-4d03-b0be-ef6e8f2df054 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:28.353003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:28.353899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:28.355031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:28.355898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b898d-e2af-432b-9e2d-f4c12d3a16a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:28.355898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b898d-e2af-432b-9e2d-f4c12d3a16a7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.356045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 284/570] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:28.356045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e909c47-fc40-4d03-b0be-ef6e8f2df054 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.367220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 287/571] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:28.368065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b898d-e2af-432b-9e2d-f4c12d3a16a7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.454503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:28.457048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:28.457365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e909c47-fc40-4d03-b0be-ef6e8f2df054 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.467586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b898d-e2af-432b-9e2d-f4c12d3a16a7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.491078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a9355-4b86-4e95-ace1-db4b9019d45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.491892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a9355-4b86-4e95-ace1-db4b9019d45f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.492167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a9355-4b86-4e95-ace1-db4b9019d45f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.492438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a9355-4b86-4e95-ace1-db4b9019d45f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.502633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a9355-4b86-4e95-ace1-db4b9019d45f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:28.503216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941bf8d-d6e4-4bb3-8346-9c1d32e1ddaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.503910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941bf8d-d6e4-4bb3-8346-9c1d32e1ddaf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.504193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941bf8d-d6e4-4bb3-8346-9c1d32e1ddaf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.504468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941bf8d-d6e4-4bb3-8346-9c1d32e1ddaf None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.505302 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941bf8d-d6e4-4bb3-8346-9c1d32e1ddaf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:28.505617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941bf8d-d6e4-4bb3-8346-9c1d32e1ddaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:28.507075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941bf8d-d6e4-4bb3-8346-9c1d32e1ddaf None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-8d8a9355-4b86-4e95-ace1-db4b9019d45f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:28.510239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:28.511141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:28.512013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 285/572] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:28.512912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a9355-4b86-4e95-ace1-db4b9019d45f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.524262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 288/573] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:28.525356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941bf8d-d6e4-4bb3-8346-9c1d32e1ddaf None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.616649 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a9355-4b86-4e95-ace1-db4b9019d45f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.620742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941bf8d-d6e4-4bb3-8346-9c1d32e1ddaf None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.650723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022600-717e-43ae-99d4-cc50066b62c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.651409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022600-717e-43ae-99d4-cc50066b62c1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.651709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022600-717e-43ae-99d4-cc50066b62c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.651979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022600-717e-43ae-99d4-cc50066b62c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.652692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1022600-717e-43ae-99d4-cc50066b62c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:28.652991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1022600-717e-43ae-99d4-cc50066b62c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:28.653123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba131a5-c061-4950-af5e-1f0774acfdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.653793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba131a5-c061-4950-af5e-1f0774acfdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.654074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba131a5-c061-4950-af5e-1f0774acfdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.654346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba131a5-c061-4950-af5e-1f0774acfdf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.654495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1022600-717e-43ae-99d4-cc50066b62c1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.655079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba131a5-c061-4950-af5e-1f0774acfdf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:28.655413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba131a5-c061-4950-af5e-1f0774acfdf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:28.656776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba131a5-c061-4950-af5e-1f0774acfdf3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.669473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 289/574] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:28.669473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba131a5-c061-4950-af5e-1f0774acfdf3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.669735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 286/575] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:28.669735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022600-717e-43ae-99d4-cc50066b62c1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.684031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:28.692021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:28.771848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba131a5-c061-4950-af5e-1f0774acfdf3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.772303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022600-717e-43ae-99d4-cc50066b62c1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.804085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950166a-fcd4-43b3-af72-50b797806a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.804535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80c44b-5150-4995-90c0-1497bfbd6df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.804732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950166a-fcd4-43b3-af72-50b797806a71 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.805010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950166a-fcd4-43b3-af72-50b797806a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.805192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c44b-5150-4995-90c0-1497bfbd6df0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.805291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950166a-fcd4-43b3-af72-50b797806a71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.805472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c44b-5150-4995-90c0-1497bfbd6df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.805744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80c44b-5150-4995-90c0-1497bfbd6df0 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.806118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950166a-fcd4-43b3-af72-50b797806a71 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=f37a95ba40774cfd89e6f397afa301a9) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:28.806420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950166a-fcd4-43b3-af72-50b797806a71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:28.806934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80c44b-5150-4995-90c0-1497bfbd6df0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:28.807249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80c44b-5150-4995-90c0-1497bfbd6df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:28.808199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950166a-fcd4-43b3-af72-50b797806a71 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.808615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80c44b-5150-4995-90c0-1497bfbd6df0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.813687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 290/576] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/role_assignments?user.id=f37a95ba40774cfd89e6f397afa301a9&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:28.814463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950166a-fcd4-43b3-af72-50b797806a71 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.823025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 287/577] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:28.824384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80c44b-5150-4995-90c0-1497bfbd6df0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:28.909187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:28.919217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:28.938597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950166a-fcd4-43b3-af72-50b797806a71 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.943975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80c44b-5150-4995-90c0-1497bfbd6df0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:28.981308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef9d3a-e7d6-423d-b21f-8e96d7c8d39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.982109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef9d3a-e7d6-423d-b21f-8e96d7c8d39f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.982389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef9d3a-e7d6-423d-b21f-8e96d7c8d39f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.982662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef9d3a-e7d6-423d-b21f-8e96d7c8d39f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.985098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee62ea-3a46-4711-bc62-41871a15f4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:28.986211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee62ea-3a46-4711-bc62-41871a15f4fb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:28.986496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee62ea-3a46-4711-bc62-41871a15f4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:28.986773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee62ea-3a46-4711-bc62-41871a15f4fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:28.993414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef9d3a-e7d6-423d-b21f-8e96d7c8d39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:28.994922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef9d3a-e7d6-423d-b21f-8e96d7c8d39f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:28.999010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee62ea-3a46-4711-bc62-41871a15f4fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.000501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee62ea-3a46-4711-bc62-41871a15f4fb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8eef9d3a-e7d6-423d-b21f-8e96d7c8d39f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:29.014881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:29.016042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:29.016903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:29.016903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:29.016903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:29.016903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:29.019622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 291/578] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6fee62ea-3a46-4711-bc62-41871a15f4fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:29.019982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:29.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:29.021910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:29.021910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:29.021910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:29.021910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:29.023025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef9d3a-e7d6-423d-b21f-8e96d7c8d39f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.025889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 288/579] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:29.026574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee62ea-3a46-4711-bc62-41871a15f4fb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.133185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee62ea-3a46-4711-bc62-41871a15f4fb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.143537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef9d3a-e7d6-423d-b21f-8e96d7c8d39f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.169107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbc777-276c-4fe6-b924-7ae7a897c759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.169899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbc777-276c-4fe6-b924-7ae7a897c759 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.170083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:29.170179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbc777-276c-4fe6-b924-7ae7a897c759 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.170439 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbc777-276c-4fe6-b924-7ae7a897c759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.173362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:29.180775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323150f-b82f-4e22-8bd2-2be68de39dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.181943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323150f-b82f-4e22-8bd2-2be68de39dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.182242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323150f-b82f-4e22-8bd2-2be68de39dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.182492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323150f-b82f-4e22-8bd2-2be68de39dc6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.183675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbc777-276c-4fe6-b924-7ae7a897c759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.185149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbc777-276c-4fe6-b924-7ae7a897c759 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.190357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323150f-b82f-4e22-8bd2-2be68de39dc6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.191828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323150f-b82f-4e22-8bd2-2be68de39dc6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6dbbc777-276c-4fe6-b924-7ae7a897c759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:29.202846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:29.203779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:29.204651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:29.204651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:29.204651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:29.204651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c323150f-b82f-4e22-8bd2-2be68de39dc6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:29.206524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:29.207403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:29.208339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 292/580] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:29.209773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323150f-b82f-4e22-8bd2-2be68de39dc6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.211062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 289/581] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:29.211823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbc777-276c-4fe6-b924-7ae7a897c759 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.314681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323150f-b82f-4e22-8bd2-2be68de39dc6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.330808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbc777-276c-4fe6-b924-7ae7a897c759 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.349079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf2a23-1c0a-4f45-a96e-43d3acd39fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.349745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf2a23-1c0a-4f45-a96e-43d3acd39fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.350030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf2a23-1c0a-4f45-a96e-43d3acd39fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.350304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cf2a23-1c0a-4f45-a96e-43d3acd39fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.351481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf2a23-1c0a-4f45-a96e-43d3acd39fd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:29.351830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf2a23-1c0a-4f45-a96e-43d3acd39fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.353578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cf2a23-1c0a-4f45-a96e-43d3acd39fd9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.365541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c50fe0-dbd9-42d4-bcc1-736dbee7a6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.366372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c50fe0-dbd9-42d4-bcc1-736dbee7a6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.366753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c50fe0-dbd9-42d4-bcc1-736dbee7a6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.367145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c50fe0-dbd9-42d4-bcc1-736dbee7a6c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.368015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c50fe0-dbd9-42d4-bcc1-736dbee7a6c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:29.368478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c50fe0-dbd9-42d4-bcc1-736dbee7a6c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.370177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c50fe0-dbd9-42d4-bcc1-736dbee7a6c8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.377277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 293/582] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:29.378779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf2a23-1c0a-4f45-a96e-43d3acd39fd9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.383209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 290/583] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:29.384185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c50fe0-dbd9-42d4-bcc1-736dbee7a6c8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.434044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:29.437723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:29.493077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cf2a23-1c0a-4f45-a96e-43d3acd39fd9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.499617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c50fe0-dbd9-42d4-bcc1-736dbee7a6c8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.528039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89243b01-5daa-420b-961c-d96856f62242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.528795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89243b01-5daa-420b-961c-d96856f62242 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.529140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89243b01-5daa-420b-961c-d96856f62242 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.529587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89243b01-5daa-420b-961c-d96856f62242 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.530473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89243b01-5daa-420b-961c-d96856f62242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:29.530839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89243b01-5daa-420b-961c-d96856f62242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.532306 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89243b01-5daa-420b-961c-d96856f62242 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.534982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541731a5-039d-4276-8ef5-cb4eb42503a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.535740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541731a5-039d-4276-8ef5-cb4eb42503a1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.536091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541731a5-039d-4276-8ef5-cb4eb42503a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.536434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541731a5-039d-4276-8ef5-cb4eb42503a1 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.537328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541731a5-039d-4276-8ef5-cb4eb42503a1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:29.537706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541731a5-039d-4276-8ef5-cb4eb42503a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.539182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541731a5-039d-4276-8ef5-cb4eb42503a1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.544986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 294/584] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:29.546008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89243b01-5daa-420b-961c-d96856f62242 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.561317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 291/585] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:29.562554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541731a5-039d-4276-8ef5-cb4eb42503a1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.659240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89243b01-5daa-420b-961c-d96856f62242 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.665025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541731a5-039d-4276-8ef5-cb4eb42503a1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.666358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:29.668391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:29.695935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc584c-3797-4f2f-aeb0-f6dcefb1b2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.696808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc584c-3797-4f2f-aeb0-f6dcefb1b2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.697079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc584c-3797-4f2f-aeb0-f6dcefb1b2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.697447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc584c-3797-4f2f-aeb0-f6dcefb1b2c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.701561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cbcfa-830b-4388-8755-cc493fbae3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.702446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cbcfa-830b-4388-8755-cc493fbae3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.702820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cbcfa-830b-4388-8755-cc493fbae3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.703193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cbcfa-830b-4388-8755-cc493fbae3b7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.704315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc584c-3797-4f2f-aeb0-f6dcefb1b2c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.705774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc584c-3797-4f2f-aeb0-f6dcefb1b2c8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.710373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cbcfa-830b-4388-8755-cc493fbae3b7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.711862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cbcfa-830b-4388-8755-cc493fbae3b7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-33dc584c-3797-4f2f-aeb0-f6dcefb1b2c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:29.720474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:29.721376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:29.722271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:29.722271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:29.722271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:29.722271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:29.722271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:29.722271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:29.722271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:29.722271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:29.722271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 295/586] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:29.722762 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc584c-3797-4f2f-aeb0-f6dcefb1b2c8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-060cbcfa-830b-4388-8755-cc493fbae3b7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:29.731073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:29.732064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:29.732993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 292/587] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:29.734791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cbcfa-830b-4388-8755-cc493fbae3b7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.834191 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc584c-3797-4f2f-aeb0-f6dcefb1b2c8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.835821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cbcfa-830b-4388-8755-cc493fbae3b7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:29.869665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cf647-b359-42a5-8b31-4bfdc8b9f0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.870469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cf647-b359-42a5-8b31-4bfdc8b9f0be None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.870746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cf647-b359-42a5-8b31-4bfdc8b9f0be None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.871022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1cf647-b359-42a5-8b31-4bfdc8b9f0be None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.872718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dab6c-635d-4e56-a289-9cf64769a451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:29.873517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dab6c-635d-4e56-a289-9cf64769a451 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:29.873795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dab6c-635d-4e56-a289-9cf64769a451 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:29.874074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dab6c-635d-4e56-a289-9cf64769a451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:29.878691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1cf647-b359-42a5-8b31-4bfdc8b9f0be None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.880309 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1cf647-b359-42a5-8b31-4bfdc8b9f0be None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.884576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dab6c-635d-4e56-a289-9cf64769a451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:29.886023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dab6c-635d-4e56-a289-9cf64769a451 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ac1cf647-b359-42a5-8b31-4bfdc8b9f0be None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:29.896727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:29.897655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 296/588] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:29.899262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cf647-b359-42a5-8b31-4bfdc8b9f0be None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6a5dab6c-635d-4e56-a289-9cf64769a451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:29.908169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:29.909046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:29.909958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:29.909958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:29.909958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:29.909958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:29.909958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 293/589] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:29.910240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dab6c-635d-4e56-a289-9cf64769a451 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:29.930920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:29.931159 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:30.011527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1cf647-b359-42a5-8b31-4bfdc8b9f0be None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.017987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dab6c-635d-4e56-a289-9cf64769a451 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.047367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38b07e-bc9a-4b26-a1d1-3aec8bedb8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.048034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38b07e-bc9a-4b26-a1d1-3aec8bedb8eb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.048314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38b07e-bc9a-4b26-a1d1-3aec8bedb8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.048625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38b07e-bc9a-4b26-a1d1-3aec8bedb8eb None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.049376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38b07e-bc9a-4b26-a1d1-3aec8bedb8eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:30.049720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38b07e-bc9a-4b26-a1d1-3aec8bedb8eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.051157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38b07e-bc9a-4b26-a1d1-3aec8bedb8eb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.053225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f3af-0c1a-4ffd-ad5a-00c50a23bc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.053881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f3af-0c1a-4ffd-ad5a-00c50a23bc19 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.054163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f3af-0c1a-4ffd-ad5a-00c50a23bc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.054440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421f3af-0c1a-4ffd-ad5a-00c50a23bc19 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.055334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421f3af-0c1a-4ffd-ad5a-00c50a23bc19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:30.055655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421f3af-0c1a-4ffd-ad5a-00c50a23bc19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.057681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421f3af-0c1a-4ffd-ad5a-00c50a23bc19 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.068866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 297/590] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:30.068866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38b07e-bc9a-4b26-a1d1-3aec8bedb8eb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.081817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 294/591] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Jul 2 03:21:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:30.082533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f3af-0c1a-4ffd-ad5a-00c50a23bc19 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.164678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:30.168724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:30.185492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38b07e-bc9a-4b26-a1d1-3aec8bedb8eb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.191992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421f3af-0c1a-4ffd-ad5a-00c50a23bc19 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.221990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3147c-f571-4a54-a153-915726e9b90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.222668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3147c-f571-4a54-a153-915726e9b90c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.222948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3147c-f571-4a54-a153-915726e9b90c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.223219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3147c-f571-4a54-a153-915726e9b90c None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.224082 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3147c-f571-4a54-a153-915726e9b90c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:30.224388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3147c-f571-4a54-a153-915726e9b90c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.225827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3147c-f571-4a54-a153-915726e9b90c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.226447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7707-1199-4814-b0ef-929537def00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.227145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7707-1199-4814-b0ef-929537def00e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.227609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7707-1199-4814-b0ef-929537def00e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.227939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7707-1199-4814-b0ef-929537def00e None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.229538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7707-1199-4814-b0ef-929537def00e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:30.229855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7707-1199-4814-b0ef-929537def00e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.231220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7707-1199-4814-b0ef-929537def00e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.243007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 295/592] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:30.244088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7707-1199-4814-b0ef-929537def00e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.248039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 298/593] 10.222.0.42 () {56 vars in 1139 bytes} [Sat Jul 2 03:21:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:30.249350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3147c-f571-4a54-a153-915726e9b90c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.346074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7707-1199-4814-b0ef-929537def00e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.351941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3147c-f571-4a54-a153-915726e9b90c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.380452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3aa13-22df-4a42-abc0-4d3b095b5788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.381345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3aa13-22df-4a42-abc0-4d3b095b5788 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.381624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3aa13-22df-4a42-abc0-4d3b095b5788 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.381896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3aa13-22df-4a42-abc0-4d3b095b5788 None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/8b7bc20034974816905978dd365993f8/roles/4a54555205fa406897a3abc7871ee1aa` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.384614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:30.387343 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c036ab-dd2d-4fb6-98e8-8b1afb2ac20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.388123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c036ab-dd2d-4fb6-98e8-8b1afb2ac20c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.388443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c036ab-dd2d-4fb6-98e8-8b1afb2ac20c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.388816 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c036ab-dd2d-4fb6-98e8-8b1afb2ac20c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.398642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3aa13-22df-4a42-abc0-4d3b095b5788 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=8b7bc20034974816905978dd365993f8, role_id=4a54555205fa406897a3abc7871ee1aa)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.399746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c036ab-dd2d-4fb6-98e8-8b1afb2ac20c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.400305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3aa13-22df-4a42-abc0-4d3b095b5788 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.401203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c036ab-dd2d-4fb6-98e8-8b1afb2ac20c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.404405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:30.423430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 296/594] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:21:30 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/8b7bc20034974816905978dd365993f8/roles/4a54555205fa406897a3abc7871ee1aa => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:30.423430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3aa13-22df-4a42-abc0-4d3b095b5788 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-83c036ab-dd2d-4fb6-98e8-8b1afb2ac20c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:30.423698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:30.424488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:30.425354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:30.425354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:30.425354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:30.425354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:30.426114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 299/595] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:30.427527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c036ab-dd2d-4fb6-98e8-8b1afb2ac20c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.531678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3aa13-22df-4a42-abc0-4d3b095b5788 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.573977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7641863-6388-42ad-9ccf-8c220ad26a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.574799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7641863-6388-42ad-9ccf-8c220ad26a72 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.575085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7641863-6388-42ad-9ccf-8c220ad26a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.575389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c036ab-dd2d-4fb6-98e8-8b1afb2ac20c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.575518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7641863-6388-42ad-9ccf-8c220ad26a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.588655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7641863-6388-42ad-9ccf-8c220ad26a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.590152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7641863-6388-42ad-9ccf-8c220ad26a72 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e7641863-6388-42ad-9ccf-8c220ad26a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:30.610978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:30.611971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:30.612838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:30.612838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:30.612838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:30.612838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:30.613082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 297/596] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:30.614207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7641863-6388-42ad-9ccf-8c220ad26a72 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.625469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607250bd-757f-4362-a52a-20d543c9fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.626281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607250bd-757f-4362-a52a-20d543c9fdfc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.626557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607250bd-757f-4362-a52a-20d543c9fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.627996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607250bd-757f-4362-a52a-20d543c9fdfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.636695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607250bd-757f-4362-a52a-20d543c9fdfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.638342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607250bd-757f-4362-a52a-20d543c9fdfc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-607250bd-757f-4362-a52a-20d543c9fdfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:30.653266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:30.654200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:30.655175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:30.655175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:30.655175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:30.655175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:30.655175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:30.655175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:30.655175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:30.655175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:30.659345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 300/597] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:30.660288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607250bd-757f-4362-a52a-20d543c9fdfc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.672504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:30.720652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:30.745149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7641863-6388-42ad-9ccf-8c220ad26a72 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.769188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607250bd-757f-4362-a52a-20d543c9fdfc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.795057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443cb84-7210-4b8c-89b0-efbd10275fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.795753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443cb84-7210-4b8c-89b0-efbd10275fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.796040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443cb84-7210-4b8c-89b0-efbd10275fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.796309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443cb84-7210-4b8c-89b0-efbd10275fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.797025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443cb84-7210-4b8c-89b0-efbd10275fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:30.797328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443cb84-7210-4b8c-89b0-efbd10275fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.798697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443cb84-7210-4b8c-89b0-efbd10275fd5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.813458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 298/598] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:30.813823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c39b8-e880-4f61-97d5-cff976b1a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.814298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443cb84-7210-4b8c-89b0-efbd10275fd5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.814492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c39b8-e880-4f61-97d5-cff976b1a186 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.814780 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c39b8-e880-4f61-97d5-cff976b1a186 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.815147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c39b8-e880-4f61-97d5-cff976b1a186 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.815912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c39b8-e880-4f61-97d5-cff976b1a186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:30.816223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c39b8-e880-4f61-97d5-cff976b1a186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.818828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c39b8-e880-4f61-97d5-cff976b1a186 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.834494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 301/599] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:30.835421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c39b8-e880-4f61-97d5-cff976b1a186 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:30.930544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:30.938361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443cb84-7210-4b8c-89b0-efbd10275fd5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.957411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c39b8-e880-4f61-97d5-cff976b1a186 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:30.973430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6b268-9c97-450e-8f31-63f82732c659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.974555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6b268-9c97-450e-8f31-63f82732c659 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.974982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6b268-9c97-450e-8f31-63f82732c659 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.975388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:30.975586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6b268-9c97-450e-8f31-63f82732c659 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:30.976607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6b268-9c97-450e-8f31-63f82732c659 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:30.976980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6b268-9c97-450e-8f31-63f82732c659 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:30.979445 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6b268-9c97-450e-8f31-63f82732c659 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:30.993237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae37ce4-b3f0-4d05-9cee-694b0a976716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:30.994227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae37ce4-b3f0-4d05-9cee-694b0a976716 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:30.994366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae37ce4-b3f0-4d05-9cee-694b0a976716 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:30.994619 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae37ce4-b3f0-4d05-9cee-694b0a976716 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.002462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae37ce4-b3f0-4d05-9cee-694b0a976716 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:31.002726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 299/600] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:31.003144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae37ce4-b3f0-4d05-9cee-694b0a976716 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:31.004090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6b268-9c97-450e-8f31-63f82732c659 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8ae37ce4-b3f0-4d05-9cee-694b0a976716 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:31.017155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:31.018210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:31.019196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:31.019196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:31.019196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:31.019196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:31.019196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:31.019196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:31.019196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:31.019196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:31.019196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 302/601] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:31.019884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe381eb-2032-4dd5-ba10-bb2aa2e247f5 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:31.020169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe381eb-2032-4dd5-ba10-bb2aa2e247f5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:31.020447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe381eb-2032-4dd5-ba10-bb2aa2e247f5 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.021851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 303/602] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:31.022371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe381eb-2032-4dd5-ba10-bb2aa2e247f5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:31.119276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6b268-9c97-450e-8f31-63f82732c659 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:31.137893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe381eb-2032-4dd5-ba10-bb2aa2e247f5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:31.154526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb5a01-079e-4198-aecb-1992edd27eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:31.155353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb5a01-079e-4198-aecb-1992edd27eec None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:31.155645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb5a01-079e-4198-aecb-1992edd27eec None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:31.155925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb5a01-079e-4198-aecb-1992edd27eec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.162735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb5a01-079e-4198-aecb-1992edd27eec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:31.164226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb5a01-079e-4198-aecb-1992edd27eec None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:31.176004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd72143-5b13-4430-83f5-bcf6a5627d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:31.177491 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd72143-5b13-4430-83f5-bcf6a5627d07 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:31.177943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd72143-5b13-4430-83f5-bcf6a5627d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-9cdb5a01-079e-4198-aecb-1992edd27eec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:31.178259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:31.179110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:31.180065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:31.180065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:31.180065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:31.180065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:31.180065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:31.180065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:31.180065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:31.180065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:31.180748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd72143-5b13-4430-83f5-bcf6a5627d07 None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/e02b8a64165541889ca929b8038412bc/roles/4a54555205fa406897a3abc7871ee1aa` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.183728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 300/603] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:31.184966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb5a01-079e-4198-aecb-1992edd27eec None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:31.199424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd72143-5b13-4430-83f5-bcf6a5627d07 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=e02b8a64165541889ca929b8038412bc, role_id=4a54555205fa406897a3abc7871ee1aa)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:31.201467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd72143-5b13-4430-83f5-bcf6a5627d07 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:31.203671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:31.219380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 304/604] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:21:31 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/e02b8a64165541889ca929b8038412bc/roles/4a54555205fa406897a3abc7871ee1aa => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:31.220673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00ee4-7a92-4d4d-8dbc-6d903c421e5c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:31.220962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00ee4-7a92-4d4d-8dbc-6d903c421e5c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:31.221243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b00ee4-7a92-4d4d-8dbc-6d903c421e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.241608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b5b00ee4-7a92-4d4d-8dbc-6d903c421e5c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:31.243206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b5b00ee4-7a92-4d4d-8dbc-6d903c421e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:31.266609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:31.321756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb5a01-079e-4198-aecb-1992edd27eec None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:31.343716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 305/605] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:31.344715 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b00ee4-7a92-4d4d-8dbc-6d903c421e5c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:31.371853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f212-0b9f-4e22-92ef-d935664ed637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:31.373859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f212-0b9f-4e22-92ef-d935664ed637 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:31.373859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f212-0b9f-4e22-92ef-d935664ed637 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:31.373859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76f212-0b9f-4e22-92ef-d935664ed637 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.374362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76f212-0b9f-4e22-92ef-d935664ed637 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:31.374680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76f212-0b9f-4e22-92ef-d935664ed637 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:31.377031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76f212-0b9f-4e22-92ef-d935664ed637 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:31.395877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 301/606] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:31.395877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f212-0b9f-4e22-92ef-d935664ed637 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:31.476652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b00ee4-7a92-4d4d-8dbc-6d903c421e5c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:31.499714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:31.527011 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73efe-549f-40d4-aeae-1a2dfa0dbe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:31.528732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73efe-549f-40d4-aeae-1a2dfa0dbe79 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:31.529482 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73efe-549f-40d4-aeae-1a2dfa0dbe79 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:31.529952 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73efe-549f-40d4-aeae-1a2dfa0dbe79 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.530955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d73efe-549f-40d4-aeae-1a2dfa0dbe79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:31.531187 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76f212-0b9f-4e22-92ef-d935664ed637 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:31.531700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d73efe-549f-40d4-aeae-1a2dfa0dbe79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:31.534462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d73efe-549f-40d4-aeae-1a2dfa0dbe79 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:31.552279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 306/607] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:31.553537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73efe-549f-40d4-aeae-1a2dfa0dbe79 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:31.570049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007786d-8ac5-4a98-adec-d2f214bde845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:31.570870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007786d-8ac5-4a98-adec-d2f214bde845 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:31.571127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007786d-8ac5-4a98-adec-d2f214bde845 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:31.571722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007786d-8ac5-4a98-adec-d2f214bde845 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.578835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:31.586863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007786d-8ac5-4a98-adec-d2f214bde845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c007786d-8ac5-4a98-adec-d2f214bde845 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:31.595421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:31.596363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 302/608] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:31.597874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007786d-8ac5-4a98-adec-d2f214bde845 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:31.698773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73efe-549f-40d4-aeae-1a2dfa0dbe79 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:31.711771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007786d-8ac5-4a98-adec-d2f214bde845 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:31.744895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a7e10-8225-40f5-b3a0-d46c56f982ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:31.746172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a7e10-8225-40f5-b3a0-d46c56f982ff None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:31.746648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a7e10-8225-40f5-b3a0-d46c56f982ff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:31.747093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a7e10-8225-40f5-b3a0-d46c56f982ff None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/2afd25850dea40568271538793e861c3/roles/4a54555205fa406897a3abc7871ee1aa` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.748075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c8dd0-091c-48c3-b0a9-8c081cc80d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:31.749096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c8dd0-091c-48c3-b0a9-8c081cc80d67 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:31.749449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c8dd0-091c-48c3-b0a9-8c081cc80d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:31.749851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c8dd0-091c-48c3-b0a9-8c081cc80d67 None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/d67be26446324c91b956e9d930e04b4d/roles/4a54555205fa406897a3abc7871ee1aa` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:31.754190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:31.767235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c8dd0-091c-48c3-b0a9-8c081cc80d67 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=d67be26446324c91b956e9d930e04b4d, role_id=4a54555205fa406897a3abc7871ee1aa)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:31.768839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c8dd0-091c-48c3-b0a9-8c081cc80d67 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:31.769173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a7e10-8225-40f5-b3a0-d46c56f982ff None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=2afd25850dea40568271538793e861c3, role_id=4a54555205fa406897a3abc7871ee1aa)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:31.770787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a7e10-8225-40f5-b3a0-d46c56f982ff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:31.786104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 303/609] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:21:31 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/d67be26446324c91b956e9d930e04b4d/roles/4a54555205fa406897a3abc7871ee1aa => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:31.787141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c8dd0-091c-48c3-b0a9-8c081cc80d67 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:31.795631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 307/610] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:21:31 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/2afd25850dea40568271538793e861c3/roles/4a54555205fa406897a3abc7871ee1aa => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:31.969353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c8dd0-091c-48c3-b0a9-8c081cc80d67 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:32.013439 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c588128-d9ea-4d74-823b-b168647b7823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:32.014041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c588128-d9ea-4d74-823b-b168647b7823 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:32.014329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c588128-d9ea-4d74-823b-b168647b7823 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:32.014639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c588128-d9ea-4d74-823b-b168647b7823 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:32.015459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c588128-d9ea-4d74-823b-b168647b7823 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:32.015736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c588128-d9ea-4d74-823b-b168647b7823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:32.017120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c588128-d9ea-4d74-823b-b168647b7823 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:32.026777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 304/611] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:32.032382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a7e10-8225-40f5-b3a0-d46c56f982ff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:32.130712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:32.195146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a7e10-8225-40f5-b3a0-d46c56f982ff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:32.242135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bdb7b-a2ff-483b-9035-12574333de51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:32.243248 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bdb7b-a2ff-483b-9035-12574333de51 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:32.243847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bdb7b-a2ff-483b-9035-12574333de51 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:32.244266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bdb7b-a2ff-483b-9035-12574333de51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:32.257904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bdb7b-a2ff-483b-9035-12574333de51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:32.259881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70bdb7b-a2ff-483b-9035-12574333de51 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a70bdb7b-a2ff-483b-9035-12574333de51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:32.282450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:32.283573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:32.284674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:32.284674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:32.284674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:32.284674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:32.286301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 308/612] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:32.296424 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c588128-d9ea-4d74-823b-b168647b7823 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:32.356420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:32.407062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c588128-d9ea-4d74-823b-b168647b7823 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:32.447381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c94c4-2fa2-4333-a6c2-22cdb4aa0ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:32.448331 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c94c4-2fa2-4333-a6c2-22cdb4aa0ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:32.448770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c94c4-2fa2-4333-a6c2-22cdb4aa0ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:32.449187 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c94c4-2fa2-4333-a6c2-22cdb4aa0ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:32.450061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c94c4-2fa2-4333-a6c2-22cdb4aa0ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:32.450511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c94c4-2fa2-4333-a6c2-22cdb4aa0ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:32.452051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435c94c4-2fa2-4333-a6c2-22cdb4aa0ce5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:32.464935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 305/613] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:32.470313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bdb7b-a2ff-483b-9035-12574333de51 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:32.629001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bdb7b-a2ff-483b-9035-12574333de51 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:32.660547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:32.684545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7a6d3-cb30-41b1-b9fe-f87543e62d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:32.685359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7a6d3-cb30-41b1-b9fe-f87543e62d0c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:32.685639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7a6d3-cb30-41b1-b9fe-f87543e62d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:32.685912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7a6d3-cb30-41b1-b9fe-f87543e62d0c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:32.698747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7a6d3-cb30-41b1-b9fe-f87543e62d0c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:32.700283 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7a6d3-cb30-41b1-b9fe-f87543e62d0c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0ad7a6d3-cb30-41b1-b9fe-f87543e62d0c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:32.718315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:32.719391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:32.720320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 309/614] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:32.725307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c94c4-2fa2-4333-a6c2-22cdb4aa0ce5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:32.741825 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569c113-2f8f-49ba-b318-a81175984717 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:32.742201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569c113-2f8f-49ba-b318-a81175984717 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:32.742550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569c113-2f8f-49ba-b318-a81175984717 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:32.744398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 310/615] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:32.749392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9921c15-1a54-4fe8-824d-651b7487bba4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:32.749689 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9921c15-1a54-4fe8-824d-651b7487bba4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:32.749957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9921c15-1a54-4fe8-824d-651b7487bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:32.776234 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b9921c15-1a54-4fe8-824d-651b7487bba4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:32.777447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b9921c15-1a54-4fe8-824d-651b7487bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:32.906130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 311/616] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:32.919355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c94c4-2fa2-4333-a6c2-22cdb4aa0ce5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:32.932647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:32.947963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9921c15-1a54-4fe8-824d-651b7487bba4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:32.973472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0268b79-5a4b-4d1d-a4ca-e3a4d77c8307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:32.974850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0268b79-5a4b-4d1d-a4ca-e3a4d77c8307 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:32.975574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0268b79-5a4b-4d1d-a4ca-e3a4d77c8307 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:32.976241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0268b79-5a4b-4d1d-a4ca-e3a4d77c8307 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:32.977845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0268b79-5a4b-4d1d-a4ca-e3a4d77c8307 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:32.978589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0268b79-5a4b-4d1d-a4ca-e3a4d77c8307 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:32.981179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0268b79-5a4b-4d1d-a4ca-e3a4d77c8307 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:33.005956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 306/617] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Jul 2 03:21:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:33.013891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0268b79-5a4b-4d1d-a4ca-e3a4d77c8307 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:33.123844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0268b79-5a4b-4d1d-a4ca-e3a4d77c8307 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:33.150797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9921c15-1a54-4fe8-824d-651b7487bba4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:33.167728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:33.172108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d157677-4334-4768-badb-87070fd8db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:33.172916 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d157677-4334-4768-badb-87070fd8db67 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:33.173242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d157677-4334-4768-badb-87070fd8db67 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:33.173516 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d157677-4334-4768-badb-87070fd8db67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:33.198031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d157677-4334-4768-badb-87070fd8db67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:33.199566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d157677-4334-4768-badb-87070fd8db67 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:33.222758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a244be7-e734-47da-ad3c-2948cfab883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:33.223581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a244be7-e734-47da-ad3c-2948cfab883d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:33.223857 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a244be7-e734-47da-ad3c-2948cfab883d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:33.224127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a244be7-e734-47da-ad3c-2948cfab883d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5d157677-4334-4768-badb-87070fd8db67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:33.233799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:33.234898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:33.235847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:33.235847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:33.235847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:33.235847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:33.235847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 307/618] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:33.241009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d157677-4334-4768-badb-87070fd8db67 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:33.256501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a244be7-e734-47da-ad3c-2948cfab883d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:33.261179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3a244be7-e734-47da-ad3c-2948cfab883d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:33.272575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:33.274981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:33.277686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 312/619] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:33.296393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a244be7-e734-47da-ad3c-2948cfab883d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:33.362416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d157677-4334-4768-badb-87070fd8db67 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:33.398415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5fa8-6778-4289-ac80-f6d6385b2f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:33.399072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5fa8-6778-4289-ac80-f6d6385b2f96 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:33.399378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5fa8-6778-4289-ac80-f6d6385b2f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:33.399817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5fa8-6778-4289-ac80-f6d6385b2f96 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:33.400603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d5fa8-6778-4289-ac80-f6d6385b2f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:33.400906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d5fa8-6778-4289-ac80-f6d6385b2f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:33.402284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d5fa8-6778-4289-ac80-f6d6385b2f96 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:33.414631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 308/620] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:33.424524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5fa8-6778-4289-ac80-f6d6385b2f96 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:33.459066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a244be7-e734-47da-ad3c-2948cfab883d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:33.530119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:33.531111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122a16a-464d-4c4b-b856-6810250d4408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:33.532136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122a16a-464d-4c4b-b856-6810250d4408 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:33.532718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122a16a-464d-4c4b-b856-6810250d4408 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:33.533124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122a16a-464d-4c4b-b856-6810250d4408 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:33.533906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122a16a-464d-4c4b-b856-6810250d4408 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:33.534363 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122a16a-464d-4c4b-b856-6810250d4408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:33.536801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7122a16a-464d-4c4b-b856-6810250d4408 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:33.557087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 313/621] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:33.562268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5fa8-6778-4289-ac80-f6d6385b2f96 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:33.563408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122a16a-464d-4c4b-b856-6810250d4408 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:33.623558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:33.631317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20477c7f-817c-4771-b593-08492f10bffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:33.632381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20477c7f-817c-4771-b593-08492f10bffb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:33.632842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20477c7f-817c-4771-b593-08492f10bffb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:33.637704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20477c7f-817c-4771-b593-08492f10bffb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:33.648021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20477c7f-817c-4771-b593-08492f10bffb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:33.649814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20477c7f-817c-4771-b593-08492f10bffb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-20477c7f-817c-4771-b593-08492f10bffb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:33.675040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:33.676050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:33.676967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:33.676967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:33.676967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:33.676967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:33.676967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:33.676967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:33.676967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:33.676967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:33.688025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 309/622] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:33.692147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20477c7f-817c-4771-b593-08492f10bffb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:33.763533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122a16a-464d-4c4b-b856-6810250d4408 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:33.829560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b69b4-2fc3-47b2-a06a-187c09205f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:33.830659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b69b4-2fc3-47b2-a06a-187c09205f7d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:33.831088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b69b4-2fc3-47b2-a06a-187c09205f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:33.831529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b69b4-2fc3-47b2-a06a-187c09205f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:33.854036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b69b4-2fc3-47b2-a06a-187c09205f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:33.856676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b69b4-2fc3-47b2-a06a-187c09205f7d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:33.873894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20477c7f-817c-4771-b593-08492f10bffb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-261b69b4-2fc3-47b2-a06a-187c09205f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:33.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:33.882108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:33.882987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:33.882987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:33.882987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:33.882987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:33.885613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 314/623] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:33.887650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72736182-7255-4cfa-8483-322164603b8e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:33.888445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72736182-7255-4cfa-8483-322164603b8e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:33.889025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72736182-7255-4cfa-8483-322164603b8e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:33.891140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 315/624] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:33.892017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72736182-7255-4cfa-8483-322164603b8e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:33.920745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb82b3e-91c1-4d94-8639-2519389ab885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:33.921565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82b3e-91c1-4d94-8639-2519389ab885 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:33.921950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82b3e-91c1-4d94-8639-2519389ab885 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:33.922345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82b3e-91c1-4d94-8639-2519389ab885 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:33.923275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82b3e-91c1-4d94-8639-2519389ab885 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:33.923721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82b3e-91c1-4d94-8639-2519389ab885 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:33.925372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82b3e-91c1-4d94-8639-2519389ab885 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:33.934126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:33.940676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 310/625] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:33.942083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abb8be-c8e0-4a93-bc97-765e7fc232a7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:33.942741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abb8be-c8e0-4a93-bc97-765e7fc232a7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:33.943121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abb8be-c8e0-4a93-bc97-765e7fc232a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:33.960786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-93abb8be-c8e0-4a93-bc97-765e7fc232a7 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:33.962224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-93abb8be-c8e0-4a93-bc97-765e7fc232a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:34.000904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:34.014226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72736182-7255-4cfa-8483-322164603b8e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:34.047559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 311/626] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:34.048590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abb8be-c8e0-4a93-bc97-765e7fc232a7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:34.050312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e12015-52df-4729-9aef-3204008e6ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:34.051183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e12015-52df-4729-9aef-3204008e6ada None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.051582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e12015-52df-4729-9aef-3204008e6ada None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.052004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e12015-52df-4729-9aef-3204008e6ada None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.052843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e12015-52df-4729-9aef-3204008e6ada None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:34.053250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e12015-52df-4729-9aef-3204008e6ada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:34.054738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e12015-52df-4729-9aef-3204008e6ada None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:34.067503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 316/627] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:34.072721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e12015-52df-4729-9aef-3204008e6ada None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:34.173940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abb8be-c8e0-4a93-bc97-765e7fc232a7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:34.197243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e12015-52df-4729-9aef-3204008e6ada None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:34.203048 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:34.212718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dbe55-7e77-4ab0-9db4-8a1d6aff9f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:34.213716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dbe55-7e77-4ab0-9db4-8a1d6aff9f66 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.214065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dbe55-7e77-4ab0-9db4-8a1d6aff9f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.214406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dbe55-7e77-4ab0-9db4-8a1d6aff9f66 None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/f37a95ba40774cfd89e6f397afa301a9/roles/4a54555205fa406897a3abc7871ee1aa` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.230255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dbe55-7e77-4ab0-9db4-8a1d6aff9f66 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=f37a95ba40774cfd89e6f397afa301a9, role_id=4a54555205fa406897a3abc7871ee1aa)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:34.231761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dbe55-7e77-4ab0-9db4-8a1d6aff9f66 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:34.237998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7f237-e4f4-4bd2-a498-897f12ebb0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:34.239080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7f237-e4f4-4bd2-a498-897f12ebb0de None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.239464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7f237-e4f4-4bd2-a498-897f12ebb0de None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.239734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7f237-e4f4-4bd2-a498-897f12ebb0de None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.246671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 312/628] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:21:34 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/f37a95ba40774cfd89e6f397afa301a9/roles/4a54555205fa406897a3abc7871ee1aa => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:34.247521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dbe55-7e77-4ab0-9db4-8a1d6aff9f66 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:34.262106 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7f237-e4f4-4bd2-a498-897f12ebb0de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:34.264239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7f237-e4f4-4bd2-a498-897f12ebb0de None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-25b7f237-e4f4-4bd2-a498-897f12ebb0de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:34.280906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:34.282053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:34.283226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 317/629] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Jul 2 03:21:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:34.284171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26554836-e7ef-4373-9bbd-84b8c1db31a2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.284171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26554836-e7ef-4373-9bbd-84b8c1db31a2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.284171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26554836-e7ef-4373-9bbd-84b8c1db31a2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.287450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 318/630] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:34.288846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e83f8-66fd-4760-94e8-09258dc6281e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.290737 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e83f8-66fd-4760-94e8-09258dc6281e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.291027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e83f8-66fd-4760-94e8-09258dc6281e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.292489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 319/631] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:34.293300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e83f8-66fd-4760-94e8-09258dc6281e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:34.297647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:34.380760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dbe55-7e77-4ab0-9db4-8a1d6aff9f66 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:34.410137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e83f8-66fd-4760-94e8-09258dc6281e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:34.422645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e66ece-dd7e-4ca4-a339-638b2470f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:34.423481 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e66ece-dd7e-4ca4-a339-638b2470f60d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.423874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e66ece-dd7e-4ca4-a339-638b2470f60d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.424148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e66ece-dd7e-4ca4-a339-638b2470f60d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.435752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e66ece-dd7e-4ca4-a339-638b2470f60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-83e66ece-dd7e-4ca4-a339-638b2470f60d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:34.443495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:34.444594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:34.445472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 313/632] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:34.446370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc9e55-307a-4d55-99ba-58f0620172a4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.446691 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc9e55-307a-4d55-99ba-58f0620172a4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.446972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc9e55-307a-4d55-99ba-58f0620172a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.455697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bd3be-3bed-4ad8-bfbc-5c90d9840176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:34.456307 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bd3be-3bed-4ad8-bfbc-5c90d9840176 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.456590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bd3be-3bed-4ad8-bfbc-5c90d9840176 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.456865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bd3be-3bed-4ad8-bfbc-5c90d9840176 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.457701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bd3be-3bed-4ad8-bfbc-5c90d9840176 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:34.458008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bd3be-3bed-4ad8-bfbc-5c90d9840176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:34.459476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bd3be-3bed-4ad8-bfbc-5c90d9840176 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:34.464373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2fcc9e55-307a-4d55-99ba-58f0620172a4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:34.465575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2fcc9e55-307a-4d55-99ba-58f0620172a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:34.478805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 320/633] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Jul 2 03:21:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:34.480198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e489a7-720c-4883-a1e7-118ac4c91769 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.480485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e489a7-720c-4883-a1e7-118ac4c91769 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.480762 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e489a7-720c-4883-a1e7-118ac4c91769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.498009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e1e489a7-720c-4883-a1e7-118ac4c91769 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:34.499186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e1e489a7-720c-4883-a1e7-118ac4c91769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:34.504040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:34.545601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 314/634] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:34.548193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc9e55-307a-4d55-99ba-58f0620172a4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:34.557879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:34.570931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 321/635] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:34.571895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e489a7-720c-4883-a1e7-118ac4c91769 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:34.667000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc9e55-307a-4d55-99ba-58f0620172a4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:34.679174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e489a7-720c-4883-a1e7-118ac4c91769 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:34.701095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b413396-04c1-4497-b419-d0fe73d0d8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:34.701808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b413396-04c1-4497-b419-d0fe73d0d8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.702094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b413396-04c1-4497-b419-d0fe73d0d8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.702371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b413396-04c1-4497-b419-d0fe73d0d8c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.703111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b413396-04c1-4497-b419-d0fe73d0d8c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:34.703456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b413396-04c1-4497-b419-d0fe73d0d8c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:34.712542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b413396-04c1-4497-b419-d0fe73d0d8c3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:34.721165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 315/636] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:34.722084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b413396-04c1-4497-b419-d0fe73d0d8c3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:34.726257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e7453-a3d0-4ebd-b645-2ce77e70e63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:34.726920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e7453-a3d0-4ebd-b645-2ce77e70e63c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.727204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e7453-a3d0-4ebd-b645-2ce77e70e63c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.727509 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e7453-a3d0-4ebd-b645-2ce77e70e63c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.757775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:34.790246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:34.801745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e7453-a3d0-4ebd-b645-2ce77e70e63c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:34.803211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e7453-a3d0-4ebd-b645-2ce77e70e63c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:34.833943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b413396-04c1-4497-b419-d0fe73d0d8c3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:34.870662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9dda8-6012-478c-8113-3975a682deb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:34.871516 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9dda8-6012-478c-8113-3975a682deb1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:34.871798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9dda8-6012-478c-8113-3975a682deb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:34.872071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9dda8-6012-478c-8113-3975a682deb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:34.883355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9dda8-6012-478c-8113-3975a682deb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f9e9dda8-6012-478c-8113-3975a682deb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:34.892469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:34.893618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:34.894475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 316/637] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:34.895182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9dda8-6012-478c-8113-3975a682deb1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:34.916422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 322/638] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:34.917494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e7453-a3d0-4ebd-b645-2ce77e70e63c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:34.995527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:35.006606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9dda8-6012-478c-8113-3975a682deb1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.019383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:35.030095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e7453-a3d0-4ebd-b645-2ce77e70e63c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.043694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db236bd-7281-4e74-8b33-268a292a03d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.044495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db236bd-7281-4e74-8b33-268a292a03d8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.044776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db236bd-7281-4e74-8b33-268a292a03d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.045274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db236bd-7281-4e74-8b33-268a292a03d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.055020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db236bd-7281-4e74-8b33-268a292a03d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4db236bd-7281-4e74-8b33-268a292a03d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:35.063183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:35.064292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 317/639] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:35.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db236bd-7281-4e74-8b33-268a292a03d8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.072624 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70068-3196-4e84-afe2-a96e44cb670e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.073648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70068-3196-4e84-afe2-a96e44cb670e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.073931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70068-3196-4e84-afe2-a96e44cb670e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.074202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f70068-3196-4e84-afe2-a96e44cb670e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.085728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f70068-3196-4e84-afe2-a96e44cb670e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.087240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f70068-3196-4e84-afe2-a96e44cb670e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-64f70068-3196-4e84-afe2-a96e44cb670e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:35.105145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:35.106016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:35.106884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:35.106884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:35.106884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:35.106884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:35.110901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 323/640] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:35.111799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70068-3196-4e84-afe2-a96e44cb670e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.182021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db236bd-7281-4e74-8b33-268a292a03d8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.215716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59eafb-a136-4304-bb4c-9a3389e56e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.216341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59eafb-a136-4304-bb4c-9a3389e56e51 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.216628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59eafb-a136-4304-bb4c-9a3389e56e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.217042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59eafb-a136-4304-bb4c-9a3389e56e51 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.217663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59eafb-a136-4304-bb4c-9a3389e56e51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:35.217964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59eafb-a136-4304-bb4c-9a3389e56e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.219369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59eafb-a136-4304-bb4c-9a3389e56e51 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.220072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f70068-3196-4e84-afe2-a96e44cb670e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.227831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 318/641] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:35.228466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59eafb-a136-4304-bb4c-9a3389e56e51 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.244156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:35.261035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7b43a-0b93-40b2-871f-0801db42976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.261774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7b43a-0b93-40b2-871f-0801db42976c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.262055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7b43a-0b93-40b2-871f-0801db42976c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.262320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7b43a-0b93-40b2-871f-0801db42976c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.273138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7b43a-0b93-40b2-871f-0801db42976c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.274837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7b43a-0b93-40b2-871f-0801db42976c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.276618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3ba7b43a-0b93-40b2-871f-0801db42976c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:35.294224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:35.295216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:35.296163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:35.296163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:35.296163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:35.296163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:35.296163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 324/642] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:35.296463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7b43a-0b93-40b2-871f-0801db42976c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.340797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59eafb-a136-4304-bb4c-9a3389e56e51 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.378529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295ac09-3c36-4153-9778-2f70e9c28de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.379205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ac09-3c36-4153-9778-2f70e9c28de5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.379524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ac09-3c36-4153-9778-2f70e9c28de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.379806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ac09-3c36-4153-9778-2f70e9c28de5 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.380551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295ac09-3c36-4153-9778-2f70e9c28de5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:35.380861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295ac09-3c36-4153-9778-2f70e9c28de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.382244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295ac09-3c36-4153-9778-2f70e9c28de5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.390802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 319/643] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:35.392612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295ac09-3c36-4153-9778-2f70e9c28de5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.404601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7b43a-0b93-40b2-871f-0801db42976c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.440373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92528b51-0313-4350-a39d-6b8c7b7c4ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.441046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92528b51-0313-4350-a39d-6b8c7b7c4ebd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.441332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92528b51-0313-4350-a39d-6b8c7b7c4ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.441604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92528b51-0313-4350-a39d-6b8c7b7c4ebd None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.442324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92528b51-0313-4350-a39d-6b8c7b7c4ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:35.442629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92528b51-0313-4350-a39d-6b8c7b7c4ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.444037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92528b51-0313-4350-a39d-6b8c7b7c4ebd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.455902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 325/644] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:35.460454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92528b51-0313-4350-a39d-6b8c7b7c4ebd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.476559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:35.504095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295ac09-3c36-4153-9778-2f70e9c28de5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.516687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:35.541155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be15e8-37ac-4d5e-a5ea-d620dabe2838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.541952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be15e8-37ac-4d5e-a5ea-d620dabe2838 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.542241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be15e8-37ac-4d5e-a5ea-d620dabe2838 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.542520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be15e8-37ac-4d5e-a5ea-d620dabe2838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.553903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be15e8-37ac-4d5e-a5ea-d620dabe2838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.555784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be15e8-37ac-4d5e-a5ea-d620dabe2838 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.569976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92528b51-0313-4350-a39d-6b8c7b7c4ebd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-09be15e8-37ac-4d5e-a5ea-d620dabe2838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:35.574743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:35.575665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:35.581294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:35.581294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:35.581294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:35.581294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:35.581294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 320/645] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:35.581294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be15e8-37ac-4d5e-a5ea-d620dabe2838 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.609594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640b028-7984-4bb2-9918-d6c43317c4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.610350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b028-7984-4bb2-9918-d6c43317c4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.610634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b028-7984-4bb2-9918-d6c43317c4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.610911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b028-7984-4bb2-9918-d6c43317c4b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.611655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640b028-7984-4bb2-9918-d6c43317c4b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:35.611954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640b028-7984-4bb2-9918-d6c43317c4b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.613327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640b028-7984-4bb2-9918-d6c43317c4b8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.626007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 326/646] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:35.626823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640b028-7984-4bb2-9918-d6c43317c4b8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.687398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be15e8-37ac-4d5e-a5ea-d620dabe2838 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.712807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:35.725253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057537fe-5cdb-424e-98c4-d9c617e2b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.726329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057537fe-5cdb-424e-98c4-d9c617e2b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.726982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057537fe-5cdb-424e-98c4-d9c617e2b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.727415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057537fe-5cdb-424e-98c4-d9c617e2b3d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.732321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640b028-7984-4bb2-9918-d6c43317c4b8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.741182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057537fe-5cdb-424e-98c4-d9c617e2b3d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.743570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057537fe-5cdb-424e-98c4-d9c617e2b3d2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.746819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-057537fe-5cdb-424e-98c4-d9c617e2b3d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:35.762102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:35.763036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:35.763993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:35.763993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:35.763993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:35.763993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:35.763993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 321/647] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:35.764287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057537fe-5cdb-424e-98c4-d9c617e2b3d2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.772556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6e29e-bcac-4a96-aa09-c002de04ab60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.773286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6e29e-bcac-4a96-aa09-c002de04ab60 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.773568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6e29e-bcac-4a96-aa09-c002de04ab60 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.773845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6e29e-bcac-4a96-aa09-c002de04ab60 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.782807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6e29e-bcac-4a96-aa09-c002de04ab60 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.784292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6e29e-bcac-4a96-aa09-c002de04ab60 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3cf6e29e-bcac-4a96-aa09-c002de04ab60 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:35.798698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:35.799751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 327/648] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:35.800706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6e29e-bcac-4a96-aa09-c002de04ab60 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.872685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057537fe-5cdb-424e-98c4-d9c617e2b3d2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.908427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34e748-6599-4c9d-8183-6e5f67a4c22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.909120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34e748-6599-4c9d-8183-6e5f67a4c22b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.909372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34e748-6599-4c9d-8183-6e5f67a4c22b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.909644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34e748-6599-4c9d-8183-6e5f67a4c22b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.910368 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34e748-6599-4c9d-8183-6e5f67a4c22b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:35.910678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34e748-6599-4c9d-8183-6e5f67a4c22b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.910909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6e29e-bcac-4a96-aa09-c002de04ab60 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:35.912098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34e748-6599-4c9d-8183-6e5f67a4c22b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.929005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 322/649] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:35.929005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34e748-6599-4c9d-8183-6e5f67a4c22b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:35.953758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450da64-c20d-454d-880b-565b8831a86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:35.954557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450da64-c20d-454d-880b-565b8831a86b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:35.954841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450da64-c20d-454d-880b-565b8831a86b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:35.955098 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450da64-c20d-454d-880b-565b8831a86b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:35.962681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450da64-c20d-454d-880b-565b8831a86b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:35.964329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450da64-c20d-454d-880b-565b8831a86b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:35.971560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2450da64-c20d-454d-880b-565b8831a86b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:35.979141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:35.980137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:35.981019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:35.981019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:35.981019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:35.981019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:35.981019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:35.981019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:35.981019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:35.981019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:35.981019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 328/650] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:35.981491 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450da64-c20d-454d-880b-565b8831a86b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.026705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:36.039802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34e748-6599-4c9d-8183-6e5f67a4c22b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.076171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ff32-59f8-4f35-a35a-f491504bd4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.076838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ff32-59f8-4f35-a35a-f491504bd4ef None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.077119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ff32-59f8-4f35-a35a-f491504bd4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.077393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35ff32-59f8-4f35-a35a-f491504bd4ef None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.078150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ff32-59f8-4f35-a35a-f491504bd4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:36.078459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ff32-59f8-4f35-a35a-f491504bd4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.079877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35ff32-59f8-4f35-a35a-f491504bd4ef None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:36.092127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 323/651] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:36.092936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ff32-59f8-4f35-a35a-f491504bd4ef None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.103802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450da64-c20d-454d-880b-565b8831a86b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.141312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35173dfb-898e-4282-8544-6d0b3cb00d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.141994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35173dfb-898e-4282-8544-6d0b3cb00d5a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.142274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35173dfb-898e-4282-8544-6d0b3cb00d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.142545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35173dfb-898e-4282-8544-6d0b3cb00d5a None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.143517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35173dfb-898e-4282-8544-6d0b3cb00d5a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:36.143828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35173dfb-898e-4282-8544-6d0b3cb00d5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.145396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35173dfb-898e-4282-8544-6d0b3cb00d5a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:36.168448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 329/652] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Jul 2 03:21:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:36.168448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35173dfb-898e-4282-8544-6d0b3cb00d5a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.200056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:36.204353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35ff32-59f8-4f35-a35a-f491504bd4ef None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.238300 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a0e8b-fcb4-4b12-a5af-adbbfd5147cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.239102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a0e8b-fcb4-4b12-a5af-adbbfd5147cc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.239413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a0e8b-fcb4-4b12-a5af-adbbfd5147cc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.239739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a0e8b-fcb4-4b12-a5af-adbbfd5147cc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.247328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a0e8b-fcb4-4b12-a5af-adbbfd5147cc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.248779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a0e8b-fcb4-4b12-a5af-adbbfd5147cc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:36.260861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-fb3a0e8b-fcb4-4b12-a5af-adbbfd5147cc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:36.262889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:36.264190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 324/653] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:36.265114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a0e8b-fcb4-4b12-a5af-adbbfd5147cc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.276200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35173dfb-898e-4282-8544-6d0b3cb00d5a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.309558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4f1a7-0fe4-4eb7-89f0-8134632cca0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.310139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4f1a7-0fe4-4eb7-89f0-8134632cca0e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.310415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4f1a7-0fe4-4eb7-89f0-8134632cca0e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.310658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4f1a7-0fe4-4eb7-89f0-8134632cca0e None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.311475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4f1a7-0fe4-4eb7-89f0-8134632cca0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:36.311780 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4f1a7-0fe4-4eb7-89f0-8134632cca0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.313211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4f1a7-0fe4-4eb7-89f0-8134632cca0e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:36.324947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 330/654] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:36.325514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4f1a7-0fe4-4eb7-89f0-8134632cca0e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.366954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a0e8b-fcb4-4b12-a5af-adbbfd5147cc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.399914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e5b3a-a8ba-4788-90f1-0765cacc8be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.400704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e5b3a-a8ba-4788-90f1-0765cacc8be5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.400982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e5b3a-a8ba-4788-90f1-0765cacc8be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.401287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e5b3a-a8ba-4788-90f1-0765cacc8be5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.409040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e5b3a-a8ba-4788-90f1-0765cacc8be5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.410485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e5b3a-a8ba-4788-90f1-0765cacc8be5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:36.423949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4f1a7-0fe4-4eb7-89f0-8134632cca0e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6a6e5b3a-a8ba-4788-90f1-0765cacc8be5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:36.424355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:36.425378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 325/655] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:36.426324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e5b3a-a8ba-4788-90f1-0765cacc8be5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.445111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:36.461272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0344de-1175-4928-91bd-2f04005255c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.461925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0344de-1175-4928-91bd-2f04005255c8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.462266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0344de-1175-4928-91bd-2f04005255c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.462541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0344de-1175-4928-91bd-2f04005255c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.483097 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:36.524424 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e5b3a-a8ba-4788-90f1-0765cacc8be5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.528620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0344de-1175-4928-91bd-2f04005255c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.530088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0344de-1175-4928-91bd-2f04005255c8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:36.557630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c8854-9a6e-4e40-821c-f28d7c5e0d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.558295 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c8854-9a6e-4e40-821c-f28d7c5e0d38 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.558582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c8854-9a6e-4e40-821c-f28d7c5e0d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.558858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c8854-9a6e-4e40-821c-f28d7c5e0d38 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.559787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c8854-9a6e-4e40-821c-f28d7c5e0d38 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:36.560045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c8854-9a6e-4e40-821c-f28d7c5e0d38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.561498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c8854-9a6e-4e40-821c-f28d7c5e0d38 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:36.578976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 326/656] 10.222.0.42 () {56 vars in 1139 bytes} [Sat Jul 2 03:21:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:36.580266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bffa1-8e2f-4497-aad7-6e40bfbf1443 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.580560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bffa1-8e2f-4497-aad7-6e40bfbf1443 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.580836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bffa1-8e2f-4497-aad7-6e40bfbf1443 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.585358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 327/657] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:36.585358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bffa1-8e2f-4497-aad7-6e40bfbf1443 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.628956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 331/658] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:36.629798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0344de-1175-4928-91bd-2f04005255c8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.675447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:36.683993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:36.700545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bffa1-8e2f-4497-aad7-6e40bfbf1443 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.730074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0344de-1175-4928-91bd-2f04005255c8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.735729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea42c68-c834-4e81-a0eb-e55b12fa27e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.736396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea42c68-c834-4e81-a0eb-e55b12fa27e1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.736678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea42c68-c834-4e81-a0eb-e55b12fa27e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.736953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea42c68-c834-4e81-a0eb-e55b12fa27e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.737883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea42c68-c834-4e81-a0eb-e55b12fa27e1 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=8b7bc20034974816905978dd365993f8) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:36.738201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea42c68-c834-4e81-a0eb-e55b12fa27e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.739690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea42c68-c834-4e81-a0eb-e55b12fa27e1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:36.746143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 328/659] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:36 2022] GET /identity/v3/role_assignments?user.id=8b7bc20034974816905978dd365993f8&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:36.746959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea42c68-c834-4e81-a0eb-e55b12fa27e1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.789412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0a911-e8d5-45c8-a65e-5ae1062f226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.790236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0a911-e8d5-45c8-a65e-5ae1062f226e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.790591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0a911-e8d5-45c8-a65e-5ae1062f226e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.791079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0a911-e8d5-45c8-a65e-5ae1062f226e None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.792063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0a911-e8d5-45c8-a65e-5ae1062f226e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:36.792376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0a911-e8d5-45c8-a65e-5ae1062f226e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.793996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b0a911-e8d5-45c8-a65e-5ae1062f226e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:36.820021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 332/660] 10.222.0.42 () {56 vars in 1137 bytes} [Sat Jul 2 03:21:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:36.820021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f39732-c614-4945-a14e-ba85d788cb8e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.820021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f39732-c614-4945-a14e-ba85d788cb8e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.820021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f39732-c614-4945-a14e-ba85d788cb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.838034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-74f39732-c614-4945-a14e-ba85d788cb8e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:36.839382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-74f39732-c614-4945-a14e-ba85d788cb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:36.865429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea42c68-c834-4e81-a0eb-e55b12fa27e1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:36.907771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c4284-3780-4fc3-9596-1f19bc48b86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:36.908898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c4284-3780-4fc3-9596-1f19bc48b86f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:36.908898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c4284-3780-4fc3-9596-1f19bc48b86f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:36.909085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965c4284-3780-4fc3-9596-1f19bc48b86f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:36.914235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 333/661] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:36.915435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f39732-c614-4945-a14e-ba85d788cb8e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:36.920125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:36.971426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:36.991984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c4284-3780-4fc3-9596-1f19bc48b86f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:36.993503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965c4284-3780-4fc3-9596-1f19bc48b86f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.034388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f39732-c614-4945-a14e-ba85d788cb8e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.068516 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00388663-6178-4232-828c-bf30fad959b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.069304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00388663-6178-4232-828c-bf30fad959b9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.069579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00388663-6178-4232-828c-bf30fad959b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.069851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00388663-6178-4232-828c-bf30fad959b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.080207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00388663-6178-4232-828c-bf30fad959b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:37.081601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00388663-6178-4232-828c-bf30fad959b9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-00388663-6178-4232-828c-bf30fad959b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:37.099648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:37.100726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:37.101793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:37.101793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:37.101793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:37.101793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:37.107152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 334/662] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:37.107152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00388663-6178-4232-828c-bf30fad959b9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.109295 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 329/663] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:37.110107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c4284-3780-4fc3-9596-1f19bc48b86f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.152651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:37.208122 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:37.213518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965c4284-3780-4fc3-9596-1f19bc48b86f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.217727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00388663-6178-4232-828c-bf30fad959b9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.250365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee13c1-1156-41a3-a9a3-4a8b98c1c31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.251258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee13c1-1156-41a3-a9a3-4a8b98c1c31a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.251568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76e73d-2efc-4d7e-a123-16efc71db41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.251681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee13c1-1156-41a3-a9a3-4a8b98c1c31a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.251836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee13c1-1156-41a3-a9a3-4a8b98c1c31a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.252199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76e73d-2efc-4d7e-a123-16efc71db41b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.252476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76e73d-2efc-4d7e-a123-16efc71db41b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.252746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76e73d-2efc-4d7e-a123-16efc71db41b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.263856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee13c1-1156-41a3-a9a3-4a8b98c1c31a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c5ee13c1-1156-41a3-a9a3-4a8b98c1c31a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:37.271660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:37.272580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:37.273395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 330/664] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:37.274077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee13c1-1156-41a3-a9a3-4a8b98c1c31a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.330005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76e73d-2efc-4d7e-a123-16efc71db41b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:37.331740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76e73d-2efc-4d7e-a123-16efc71db41b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.373069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:37.377239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee13c1-1156-41a3-a9a3-4a8b98c1c31a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.414373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08516c-dc9f-4610-9893-c31415ea1866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.415034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08516c-dc9f-4610-9893-c31415ea1866 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.415452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08516c-dc9f-4610-9893-c31415ea1866 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.415933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08516c-dc9f-4610-9893-c31415ea1866 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.416666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08516c-dc9f-4610-9893-c31415ea1866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:37.416965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08516c-dc9f-4610-9893-c31415ea1866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:37.418352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08516c-dc9f-4610-9893-c31415ea1866 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.431312 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 331/665] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:37.432543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08516c-dc9f-4610-9893-c31415ea1866 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.438568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:37.449171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 335/666] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:37.450213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76e73d-2efc-4d7e-a123-16efc71db41b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.543287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08516c-dc9f-4610-9893-c31415ea1866 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.550237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76e73d-2efc-4d7e-a123-16efc71db41b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.579537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73877b5d-6131-4d99-acd9-ea976f71b3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.580631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73877b5d-6131-4d99-acd9-ea976f71b3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.580939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73877b5d-6131-4d99-acd9-ea976f71b3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.581214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73877b5d-6131-4d99-acd9-ea976f71b3b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.588207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38600196-ef7f-455c-8ec9-dc66eb189cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.588893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38600196-ef7f-455c-8ec9-dc66eb189cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.589173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38600196-ef7f-455c-8ec9-dc66eb189cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.589447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38600196-ef7f-455c-8ec9-dc66eb189cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.590190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38600196-ef7f-455c-8ec9-dc66eb189cd5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:37.590493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38600196-ef7f-455c-8ec9-dc66eb189cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:37.591889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73877b5d-6131-4d99-acd9-ea976f71b3b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:37.592014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38600196-ef7f-455c-8ec9-dc66eb189cd5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.593313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73877b5d-6131-4d99-acd9-ea976f71b3b7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.600314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 336/667] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:37.601086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38600196-ef7f-455c-8ec9-dc66eb189cd5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-73877b5d-6131-4d99-acd9-ea976f71b3b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:37.615504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:37.616486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:37.617398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:37.617398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:37.617398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:37.617398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:37.617398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 332/668] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:37.618371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73877b5d-6131-4d99-acd9-ea976f71b3b7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.635660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:37.659537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:37.709439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38600196-ef7f-455c-8ec9-dc66eb189cd5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.725533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73877b5d-6131-4d99-acd9-ea976f71b3b7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.746469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccf42d-f2f0-4d53-8437-e0539055db4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.747263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccf42d-f2f0-4d53-8437-e0539055db4d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.747584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccf42d-f2f0-4d53-8437-e0539055db4d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.747855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccf42d-f2f0-4d53-8437-e0539055db4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.754478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccf42d-f2f0-4d53-8437-e0539055db4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:37.756048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccf42d-f2f0-4d53-8437-e0539055db4d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.759251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344e5ce-27f4-4964-aea4-d69e7faa7a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.760104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344e5ce-27f4-4964-aea4-d69e7faa7a42 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.760384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344e5ce-27f4-4964-aea4-d69e7faa7a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.760656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344e5ce-27f4-4964-aea4-d69e7faa7a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b5ccf42d-f2f0-4d53-8437-e0539055db4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:37.770391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:37.771279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:37.772207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:37.772207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:37.772207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:37.772207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:37.772207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:37.772207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:37.772207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:37.772207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:37.773212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344e5ce-27f4-4964-aea4-d69e7faa7a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:37.773212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344e5ce-27f4-4964-aea4-d69e7faa7a42 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.776370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 337/669] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:37.777040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccf42d-f2f0-4d53-8437-e0539055db4d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e344e5ce-27f4-4964-aea4-d69e7faa7a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:37.792597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:37.793520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:37.794420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:37.794420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:37.794420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:37.794420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:37.794420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 333/670] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:37.795260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344e5ce-27f4-4964-aea4-d69e7faa7a42 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.862501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:37.884268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccf42d-f2f0-4d53-8437-e0539055db4d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.901897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:37.904441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344e5ce-27f4-4964-aea4-d69e7faa7a42 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:37.926700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaa9ed-3f81-46cd-a1d6-f73554925d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.927549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaa9ed-3f81-46cd-a1d6-f73554925d8d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.927992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaa9ed-3f81-46cd-a1d6-f73554925d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.928265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaa9ed-3f81-46cd-a1d6-f73554925d8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.940467 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d94d3-fd20-4d92-816e-d983ed731585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:37.941158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d94d3-fd20-4d92-816e-d983ed731585 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:37.941443 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d94d3-fd20-4d92-816e-d983ed731585 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:37.941721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d94d3-fd20-4d92-816e-d983ed731585 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:37.942453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d94d3-fd20-4d92-816e-d983ed731585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:37.942762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d94d3-fd20-4d92-816e-d983ed731585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:37.943457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaa9ed-3f81-46cd-a1d6-f73554925d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:37.944194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d94d3-fd20-4d92-816e-d983ed731585 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.945224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaa9ed-3f81-46cd-a1d6-f73554925d8d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:37.960088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 334/671] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:37.960088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d94d3-fd20-4d92-816e-d983ed731585 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-dcbaa9ed-3f81-46cd-a1d6-f73554925d8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:37.969580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:37.970586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:37.971501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:37.971501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:37.971501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:37.971501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:37.971501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 338/672] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:37.971772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaa9ed-3f81-46cd-a1d6-f73554925d8d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.063178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d94d3-fd20-4d92-816e-d983ed731585 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.075818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaa9ed-3f81-46cd-a1d6-f73554925d8d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.086411 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:38.098444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7579f1-f870-4812-857f-5b9455838ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.099106 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7579f1-f870-4812-857f-5b9455838ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.099416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7579f1-f870-4812-857f-5b9455838ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.099694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7579f1-f870-4812-857f-5b9455838ff2 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.100478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7579f1-f870-4812-857f-5b9455838ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:38.100793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7579f1-f870-4812-857f-5b9455838ff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.102250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7579f1-f870-4812-857f-5b9455838ff2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.119429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 335/673] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:38.119429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7579f1-f870-4812-857f-5b9455838ff2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.125001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a9d4-868e-4433-8714-b1d1a5b3dc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.125683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a9d4-868e-4433-8714-b1d1a5b3dc07 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.126370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a9d4-868e-4433-8714-b1d1a5b3dc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.126654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a9d4-868e-4433-8714-b1d1a5b3dc07 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.130583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461a9d4-868e-4433-8714-b1d1a5b3dc07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:38.130583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461a9d4-868e-4433-8714-b1d1a5b3dc07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.130583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461a9d4-868e-4433-8714-b1d1a5b3dc07 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.149017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 339/674] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:38.149017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a9d4-868e-4433-8714-b1d1a5b3dc07 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.181988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:38.226097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7579f1-f870-4812-857f-5b9455838ff2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.260953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db879428-f764-4890-a534-ba35f9403a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.261750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db879428-f764-4890-a534-ba35f9403a47 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.262033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db879428-f764-4890-a534-ba35f9403a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.262340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db879428-f764-4890-a534-ba35f9403a47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.265235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a9d4-868e-4433-8714-b1d1a5b3dc07 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.269263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db879428-f764-4890-a534-ba35f9403a47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.270871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db879428-f764-4890-a534-ba35f9403a47 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-db879428-f764-4890-a534-ba35f9403a47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:38.286108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:38.287042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:38.287962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:38.287962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:38.287962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:38.287962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:38.287962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:38.287962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:38.287962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:38.287962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:38.287962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 336/675] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:38.288447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db879428-f764-4890-a534-ba35f9403a47 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.307850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322cc10-9653-4437-ba55-6c6b1aa9983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.308798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322cc10-9653-4437-ba55-6c6b1aa9983b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.309263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322cc10-9653-4437-ba55-6c6b1aa9983b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.309533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322cc10-9653-4437-ba55-6c6b1aa9983b None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.310248 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322cc10-9653-4437-ba55-6c6b1aa9983b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:38.310548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322cc10-9653-4437-ba55-6c6b1aa9983b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.311956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8322cc10-9653-4437-ba55-6c6b1aa9983b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.328675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 340/676] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:38.328675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322cc10-9653-4437-ba55-6c6b1aa9983b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.341253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:38.399049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db879428-f764-4890-a534-ba35f9403a47 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.415938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:38.431487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322cc10-9653-4437-ba55-6c6b1aa9983b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.433644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc87819-7d2d-4946-9861-91e8374dc058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.434403 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc87819-7d2d-4946-9861-91e8374dc058 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.434681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc87819-7d2d-4946-9861-91e8374dc058 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.434953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc87819-7d2d-4946-9861-91e8374dc058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.435808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc87819-7d2d-4946-9861-91e8374dc058 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=e02b8a64165541889ca929b8038412bc) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:38.436118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc87819-7d2d-4946-9861-91e8374dc058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.437556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc87819-7d2d-4946-9861-91e8374dc058 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.455339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 337/677] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/role_assignments?user.id=e02b8a64165541889ca929b8038412bc&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:38.455339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc87819-7d2d-4946-9861-91e8374dc058 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.488839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26cd01-17ef-4175-91d8-d2e03daaf8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.490324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26cd01-17ef-4175-91d8-d2e03daaf8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.490601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26cd01-17ef-4175-91d8-d2e03daaf8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.490872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26cd01-17ef-4175-91d8-d2e03daaf8b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.500979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26cd01-17ef-4175-91d8-d2e03daaf8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.502955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26cd01-17ef-4175-91d8-d2e03daaf8b7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-6c26cd01-17ef-4175-91d8-d2e03daaf8b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:38.518617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:38.519610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:38.520483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:38.520483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:38.520483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:38.520483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:38.520483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:38.520483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:38.520483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:38.520483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:38.524745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 341/678] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:38.525713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26cd01-17ef-4175-91d8-d2e03daaf8b7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.558736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc87819-7d2d-4946-9861-91e8374dc058 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.568466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:38.594584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c1c7e-b178-4c2d-838a-0179a79416b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.595403 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c1c7e-b178-4c2d-838a-0179a79416b7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.595686 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c1c7e-b178-4c2d-838a-0179a79416b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.595958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c1c7e-b178-4c2d-838a-0179a79416b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.596779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c1c7e-b178-4c2d-838a-0179a79416b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:38.597065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c1c7e-b178-4c2d-838a-0179a79416b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.598597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c1c7e-b178-4c2d-838a-0179a79416b7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.612187 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 338/679] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:38.613585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c1c7e-b178-4c2d-838a-0179a79416b7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.643560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26cd01-17ef-4175-91d8-d2e03daaf8b7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.686403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824faef2-a80a-4916-bf45-5f8b78c636c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.687697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824faef2-a80a-4916-bf45-5f8b78c636c7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.688174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824faef2-a80a-4916-bf45-5f8b78c636c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.688648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824faef2-a80a-4916-bf45-5f8b78c636c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.697509 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824faef2-a80a-4916-bf45-5f8b78c636c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.699055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824faef2-a80a-4916-bf45-5f8b78c636c7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-824faef2-a80a-4916-bf45-5f8b78c636c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:38.715150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:38.716344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 342/680] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:38.717757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824faef2-a80a-4916-bf45-5f8b78c636c7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.729367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:38.745508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c1c7e-b178-4c2d-838a-0179a79416b7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.782234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212316fe-8ee3-4368-b2b1-56ec7824c7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.783198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212316fe-8ee3-4368-b2b1-56ec7824c7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.783688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212316fe-8ee3-4368-b2b1-56ec7824c7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.784131 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212316fe-8ee3-4368-b2b1-56ec7824c7b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.785049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212316fe-8ee3-4368-b2b1-56ec7824c7b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:38.785586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212316fe-8ee3-4368-b2b1-56ec7824c7b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.787255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212316fe-8ee3-4368-b2b1-56ec7824c7b0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.800326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 339/681] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:38.801367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212316fe-8ee3-4368-b2b1-56ec7824c7b0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.831101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:38.836871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824faef2-a80a-4916-bf45-5f8b78c636c7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.873583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1255f2-a99a-4e06-98dd-2ae075dc1efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.874349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1255f2-a99a-4e06-98dd-2ae075dc1efa None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.874727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1255f2-a99a-4e06-98dd-2ae075dc1efa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.875004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1255f2-a99a-4e06-98dd-2ae075dc1efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.875862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1255f2-a99a-4e06-98dd-2ae075dc1efa None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=d67be26446324c91b956e9d930e04b4d) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:38.876171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1255f2-a99a-4e06-98dd-2ae075dc1efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.877604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1255f2-a99a-4e06-98dd-2ae075dc1efa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:38.887366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 343/682] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/role_assignments?user.id=d67be26446324c91b956e9d930e04b4d&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:38.887366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1255f2-a99a-4e06-98dd-2ae075dc1efa None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:38.922837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212316fe-8ee3-4368-b2b1-56ec7824c7b0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:38.964790 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:38.974186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49688d2a-4962-49b8-80eb-97ab0a3330df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:38.975213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49688d2a-4962-49b8-80eb-97ab0a3330df None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:38.975793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49688d2a-4962-49b8-80eb-97ab0a3330df None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:38.976758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49688d2a-4962-49b8-80eb-97ab0a3330df None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:38.977738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49688d2a-4962-49b8-80eb-97ab0a3330df None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:38.978486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49688d2a-4962-49b8-80eb-97ab0a3330df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:38.980279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49688d2a-4962-49b8-80eb-97ab0a3330df None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:39.004075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1255f2-a99a-4e06-98dd-2ae075dc1efa None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:39.007458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 340/683] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:39.008976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e724-3d62-41d2-98c4-1c8fc3eccc51 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.009265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e724-3d62-41d2-98c4-1c8fc3eccc51 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.009552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2e724-3d62-41d2-98c4-1c8fc3eccc51 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.010900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 341/684] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:39.011639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2e724-3d62-41d2-98c4-1c8fc3eccc51 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:39.055774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c04463-eb85-4360-80a0-e564268ef701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:39.056481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c04463-eb85-4360-80a0-e564268ef701 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.057076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c04463-eb85-4360-80a0-e564268ef701 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.057346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c04463-eb85-4360-80a0-e564268ef701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.058236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c04463-eb85-4360-80a0-e564268ef701 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=2afd25850dea40568271538793e861c3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:39.058551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c04463-eb85-4360-80a0-e564268ef701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:39.060095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c04463-eb85-4360-80a0-e564268ef701 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:39.066453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 344/685] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:38 2022] GET /identity/v3/role_assignments?user.id=2afd25850dea40568271538793e861c3&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:39.071361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaca095-b9d9-4e57-8592-175fc55e6a07 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.071361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaca095-b9d9-4e57-8592-175fc55e6a07 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.071361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaca095-b9d9-4e57-8592-175fc55e6a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.096025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-acaca095-b9d9-4e57-8592-175fc55e6a07 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:39.097776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-acaca095-b9d9-4e57-8592-175fc55e6a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:39.131345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:39.153240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2e724-3d62-41d2-98c4-1c8fc3eccc51 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:39.193994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 345/686] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:39.197967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd33c9-ad40-414d-b18d-b654d51842d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:39.198638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd33c9-ad40-414d-b18d-b654d51842d4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.198921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd33c9-ad40-414d-b18d-b654d51842d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.199198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd33c9-ad40-414d-b18d-b654d51842d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.219023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaca095-b9d9-4e57-8592-175fc55e6a07 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:39.282311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd33c9-ad40-414d-b18d-b654d51842d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:39.283870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd33c9-ad40-414d-b18d-b654d51842d4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:39.300981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:39.344066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaca095-b9d9-4e57-8592-175fc55e6a07 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:39.381621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc8b7-756b-4311-aeb9-f25b30aa16cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:39.382607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc8b7-756b-4311-aeb9-f25b30aa16cf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.382982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc8b7-756b-4311-aeb9-f25b30aa16cf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.383258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc8b7-756b-4311-aeb9-f25b30aa16cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.386767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:39.398328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addc8b7-756b-4311-aeb9-f25b30aa16cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:39.407802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 342/687] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5addc8b7-756b-4311-aeb9-f25b30aa16cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:39.408064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:39.408937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:39.409753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:39.415312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 346/688] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:39.419862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd33c9-ad40-414d-b18d-b654d51842d4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:39.422702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc8b7-756b-4311-aeb9-f25b30aa16cf None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:39.540421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd33c9-ad40-414d-b18d-b654d51842d4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:39.568060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc8b7-756b-4311-aeb9-f25b30aa16cf None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:39.577374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac3e27-058f-4ac9-920b-68074eb2a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:39.578256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac3e27-058f-4ac9-920b-68074eb2a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.578983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac3e27-058f-4ac9-920b-68074eb2a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.578983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac3e27-058f-4ac9-920b-68074eb2a4d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.591462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac3e27-058f-4ac9-920b-68074eb2a4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:39.593030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac3e27-058f-4ac9-920b-68074eb2a4d4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:39.599409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f4ac3e27-058f-4ac9-920b-68074eb2a4d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:39.612758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:39.613719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:39.614603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:39.614603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:39.614603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:39.614603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:39.616194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac90fa-7feb-4e1e-8202-659be311dc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:39.617096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac90fa-7feb-4e1e-8202-659be311dc32 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.617558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac90fa-7feb-4e1e-8202-659be311dc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.618024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac90fa-7feb-4e1e-8202-659be311dc32 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.619121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac90fa-7feb-4e1e-8202-659be311dc32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:39.619665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac90fa-7feb-4e1e-8202-659be311dc32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:39.620493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 343/689] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:39.621417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac3e27-058f-4ac9-920b-68074eb2a4d4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:39.621776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac90fa-7feb-4e1e-8202-659be311dc32 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:39.633609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 347/690] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:39.639332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac90fa-7feb-4e1e-8202-659be311dc32 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:39.668381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:39.737857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac3e27-058f-4ac9-920b-68074eb2a4d4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:39.749125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac90fa-7feb-4e1e-8202-659be311dc32 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:39.782278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d8724-930d-49ab-95c5-efc21cffcad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:39.783334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d8724-930d-49ab-95c5-efc21cffcad7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.783697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d8724-930d-49ab-95c5-efc21cffcad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.784075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d8724-930d-49ab-95c5-efc21cffcad7 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.784588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3665ee-4bb1-4348-96ef-2aa9fd6f3641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:39.784997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d8724-930d-49ab-95c5-efc21cffcad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:39.785416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d8724-930d-49ab-95c5-efc21cffcad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:39.785871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3665ee-4bb1-4348-96ef-2aa9fd6f3641 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.786282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3665ee-4bb1-4348-96ef-2aa9fd6f3641 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.786692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3665ee-4bb1-4348-96ef-2aa9fd6f3641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.787185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d8724-930d-49ab-95c5-efc21cffcad7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:39.798833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3665ee-4bb1-4348-96ef-2aa9fd6f3641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:39.800594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3665ee-4bb1-4348-96ef-2aa9fd6f3641 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:39.816125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 344/691] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9b3665ee-4bb1-4348-96ef-2aa9fd6f3641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:39.819152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:39.820143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:39.821049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:39.821049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:39.821049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:39.821049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:39.822654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d8724-930d-49ab-95c5-efc21cffcad7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:39.827576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 348/692] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:39.828730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3665ee-4bb1-4348-96ef-2aa9fd6f3641 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:39.871911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:39.937340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3665ee-4bb1-4348-96ef-2aa9fd6f3641 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:39.982373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4e37e-3e9d-4e03-b681-2b15be7f7676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:39.983504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4e37e-3e9d-4e03-b681-2b15be7f7676 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:39.983999 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4e37e-3e9d-4e03-b681-2b15be7f7676 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:39.984464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4e37e-3e9d-4e03-b681-2b15be7f7676 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:39.985406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4e37e-3e9d-4e03-b681-2b15be7f7676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:39.985918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4e37e-3e9d-4e03-b681-2b15be7f7676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:39.987580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4e37e-3e9d-4e03-b681-2b15be7f7676 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:40.006810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 349/693] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:40.015669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4e37e-3e9d-4e03-b681-2b15be7f7676 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:40.037151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:40.050463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d8724-930d-49ab-95c5-efc21cffcad7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:40.154208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:40.158666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4e37e-3e9d-4e03-b681-2b15be7f7676 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:40.176595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595d37c-0ae2-4384-8b61-297565f085fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:40.177423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595d37c-0ae2-4384-8b61-297565f085fe None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:40.177713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595d37c-0ae2-4384-8b61-297565f085fe None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:40.177996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595d37c-0ae2-4384-8b61-297565f085fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:40.200447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595d37c-0ae2-4384-8b61-297565f085fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:40.201953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595d37c-0ae2-4384-8b61-297565f085fe None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:40.208094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab20ec9-1ab3-42b0-b53e-d5e73a6d89b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:40.209424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab20ec9-1ab3-42b0-b53e-d5e73a6d89b0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:40.209901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab20ec9-1ab3-42b0-b53e-d5e73a6d89b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:40.210778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab20ec9-1ab3-42b0-b53e-d5e73a6d89b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:40.218984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab20ec9-1ab3-42b0-b53e-d5e73a6d89b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:40.225114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab20ec9-1ab3-42b0-b53e-d5e73a6d89b0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-5595d37c-0ae2-4384-8b61-297565f085fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:40.238858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:40.239792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:40.240694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:40.240694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:40.240694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:40.240694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:40.240694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:40.240694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:40.240694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:40.240694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:40.240694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 345/694] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1ab20ec9-1ab3-42b0-b53e-d5e73a6d89b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:40.244838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:40.245738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:40.247352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:40.248572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595d37c-0ae2-4384-8b61-297565f085fe None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:40.252967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 350/695] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Jul 2 03:21:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:40.264919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab20ec9-1ab3-42b0-b53e-d5e73a6d89b0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:40.413089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab20ec9-1ab3-42b0-b53e-d5e73a6d89b0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:40.465305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e1463-bd28-49ba-b460-e59d3602e658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:40.467051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e1463-bd28-49ba-b460-e59d3602e658 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:40.468428 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e1463-bd28-49ba-b460-e59d3602e658 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:40.468929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e1463-bd28-49ba-b460-e59d3602e658 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:40.469957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e1463-bd28-49ba-b460-e59d3602e658 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:40.470454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e1463-bd28-49ba-b460-e59d3602e658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:40.473105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e1463-bd28-49ba-b460-e59d3602e658 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:40.504313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 351/696] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Jul 2 03:21:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:40.505552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e1463-bd28-49ba-b460-e59d3602e658 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:40.532779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:40.617308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595d37c-0ae2-4384-8b61-297565f085fe None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:40.660278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e1463-bd28-49ba-b460-e59d3602e658 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:40.737353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584be2f-4af9-461b-adc2-54355d348c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:40.738385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584be2f-4af9-461b-adc2-54355d348c3f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:40.738856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584be2f-4af9-461b-adc2-54355d348c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:40.739339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584be2f-4af9-461b-adc2-54355d348c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:40.760939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082334-69ab-48d7-ac79-32055ba04b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:40.761651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082334-69ab-48d7-ac79-32055ba04b6f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:40.761932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082334-69ab-48d7-ac79-32055ba04b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:40.762204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082334-69ab-48d7-ac79-32055ba04b6f None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:40.762992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082334-69ab-48d7-ac79-32055ba04b6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:40.763322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082334-69ab-48d7-ac79-32055ba04b6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:40.764779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082334-69ab-48d7-ac79-32055ba04b6f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:40.788464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 346/697] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 542 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:40.790272 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fe5db-1448-4efe-844b-db86c64a2fe0 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:40.790792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fe5db-1448-4efe-844b-db86c64a2fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:40.791280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9fe5db-1448-4efe-844b-db86c64a2fe0 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:40.792974 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 347/698] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:40.795015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fe5db-1448-4efe-844b-db86c64a2fe0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:40.834553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:40.851711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584be2f-4af9-461b-adc2-54355d348c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:40.853805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584be2f-4af9-461b-adc2-54355d348c3f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:40.875528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:40.924798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9fe5db-1448-4efe-844b-db86c64a2fe0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:40.960370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481629fc-78f2-4450-8273-bd0b51cae54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:40.961577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481629fc-78f2-4450-8273-bd0b51cae54f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:40.962091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481629fc-78f2-4450-8273-bd0b51cae54f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:40.962608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481629fc-78f2-4450-8273-bd0b51cae54f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:40.963763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481629fc-78f2-4450-8273-bd0b51cae54f None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=f37a95ba40774cfd89e6f397afa301a9) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:40.964289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481629fc-78f2-4450-8273-bd0b51cae54f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:40.965969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481629fc-78f2-4450-8273-bd0b51cae54f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:40.973244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 348/699] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:40 2022] GET /identity/v3/role_assignments?user.id=f37a95ba40774cfd89e6f397afa301a9&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:40.974698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b543cb-89e0-4b87-9595-c280e32d847b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:40.975343 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 352/700] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:40.980002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b543cb-89e0-4b87-9595-c280e32d847b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:40.980370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b543cb-89e0-4b87-9595-c280e32d847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:40.981116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584be2f-4af9-461b-adc2-54355d348c3f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:41.001327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b9b543cb-89e0-4b87-9595-c280e32d847b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:41.002510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b9b543cb-89e0-4b87-9595-c280e32d847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:41.074548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 349/701] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:41.125256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584be2f-4af9-461b-adc2-54355d348c3f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:41.148964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:41.160427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b543cb-89e0-4b87-9595-c280e32d847b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:41.171884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d31956-60b3-4668-ac5d-883caca9095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:41.172694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d31956-60b3-4668-ac5d-883caca9095c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.172978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d31956-60b3-4668-ac5d-883caca9095c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.173251 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d31956-60b3-4668-ac5d-883caca9095c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.186155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d31956-60b3-4668-ac5d-883caca9095c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:41.187670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d31956-60b3-4668-ac5d-883caca9095c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:41.200441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b0d31956-60b3-4668-ac5d-883caca9095c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:41.210566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:41.211458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:41.212535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:41.212535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:41.212535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:41.212535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:41.212535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 353/702] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:41.217957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d31956-60b3-4668-ac5d-883caca9095c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:41.289884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b543cb-89e0-4b87-9595-c280e32d847b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:41.329762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44155075-8ff4-4174-968a-bd367f1bc634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:41.330676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44155075-8ff4-4174-968a-bd367f1bc634 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.330962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44155075-8ff4-4174-968a-bd367f1bc634 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.331365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44155075-8ff4-4174-968a-bd367f1bc634 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.332034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44155075-8ff4-4174-968a-bd367f1bc634 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:41.333521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44155075-8ff4-4174-968a-bd367f1bc634 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:41.340957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d31956-60b3-4668-ac5d-883caca9095c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-44155075-8ff4-4174-968a-bd367f1bc634 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:41.349607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:41.351747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:41.353714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 350/703] 10.222.0.42 () {56 vars in 1129 bytes} [Sat Jul 2 03:21:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:41.360440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44155075-8ff4-4174-968a-bd367f1bc634 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:41.385714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bcc3a-afa2-4c24-bdde-23b68250638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:41.386398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bcc3a-afa2-4c24-bdde-23b68250638f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.386688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bcc3a-afa2-4c24-bdde-23b68250638f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.386971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bcc3a-afa2-4c24-bdde-23b68250638f None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.387733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bcc3a-afa2-4c24-bdde-23b68250638f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:41.388046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bcc3a-afa2-4c24-bdde-23b68250638f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:41.389481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53bcc3a-afa2-4c24-bdde-23b68250638f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:41.405928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 354/704] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:41.411137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bcc3a-afa2-4c24-bdde-23b68250638f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:41.445267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:41.456205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:41.484609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44155075-8ff4-4174-968a-bd367f1bc634 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:41.526086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d03a7-0a23-460a-b7d8-154e881b71f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:41.526764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d03a7-0a23-460a-b7d8-154e881b71f3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.527048 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d03a7-0a23-460a-b7d8-154e881b71f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.527347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d03a7-0a23-460a-b7d8-154e881b71f3 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.528060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d03a7-0a23-460a-b7d8-154e881b71f3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:41.528530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d03a7-0a23-460a-b7d8-154e881b71f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:41.530533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d03a7-0a23-460a-b7d8-154e881b71f3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:41.535530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bcc3a-afa2-4c24-bdde-23b68250638f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:41.536820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 351/705] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Jul 2 03:21:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:41.542782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d03a7-0a23-460a-b7d8-154e881b71f3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:41.577144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69f646-c220-4c3a-a98b-0f576b3ca665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:41.578347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69f646-c220-4c3a-a98b-0f576b3ca665 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.578832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69f646-c220-4c3a-a98b-0f576b3ca665 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.579392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69f646-c220-4c3a-a98b-0f576b3ca665 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.587273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69f646-c220-4c3a-a98b-0f576b3ca665 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:41.589119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69f646-c220-4c3a-a98b-0f576b3ca665 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ba69f646-c220-4c3a-a98b-0f576b3ca665 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:41.606046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:41.607017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:41.607954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:41.607954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:41.607954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:41.607954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:41.607954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:41.607954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:41.607954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:41.607954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:41.610723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 355/706] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:41.615753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69f646-c220-4c3a-a98b-0f576b3ca665 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:41.659480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d03a7-0a23-460a-b7d8-154e881b71f3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:41.707622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f038298-f136-4058-a507-88ce4dfc9cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:41.708297 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038298-f136-4058-a507-88ce4dfc9cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.708580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038298-f136-4058-a507-88ce4dfc9cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.708859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f038298-f136-4058-a507-88ce4dfc9cd5 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.709573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f038298-f136-4058-a507-88ce4dfc9cd5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:41.709878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f038298-f136-4058-a507-88ce4dfc9cd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:41.711267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f038298-f136-4058-a507-88ce4dfc9cd5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:41.724290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 352/707] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Jul 2 03:21:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:41.724290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2ba9e-39d3-4f2d-8397-9e589c123482 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.724290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2ba9e-39d3-4f2d-8397-9e589c123482 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.724290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2ba9e-39d3-4f2d-8397-9e589c123482 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.724290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 353/708] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:41.732929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1e1c7-acd9-49d6-9dc2-f3fdf01a9468 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.733267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1e1c7-acd9-49d6-9dc2-f3fdf01a9468 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.733488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1e1c7-acd9-49d6-9dc2-f3fdf01a9468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.740685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:41.754928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:41.763000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-81e1e1c7-acd9-49d6-9dc2-f3fdf01a9468 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:41.764418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-81e1e1c7-acd9-49d6-9dc2-f3fdf01a9468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:41.786856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69f646-c220-4c3a-a98b-0f576b3ca665 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:41.864441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 354/709] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:41.865819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650c48b-024b-49dd-806c-1623382665c1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.866104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650c48b-024b-49dd-806c-1623382665c1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.867648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c2330-a921-4ed8-aa64-b430619a16ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:41.868525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c2330-a921-4ed8-aa64-b430619a16ed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.868860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c2330-a921-4ed8-aa64-b430619a16ed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.869301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c2330-a921-4ed8-aa64-b430619a16ed None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.870164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c2330-a921-4ed8-aa64-b430619a16ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:41.870537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c2330-a921-4ed8-aa64-b430619a16ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:41.871456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650c48b-024b-49dd-806c-1623382665c1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.872183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c2330-a921-4ed8-aa64-b430619a16ed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:41.873281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 355/710] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:41.879949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803fdf4-9d30-4334-8bd1-92f6775f7093 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:41.880278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803fdf4-9d30-4334-8bd1-92f6775f7093 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:41.880621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803fdf4-9d30-4334-8bd1-92f6775f7093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:41.889887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 356/711] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:41.895684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c2330-a921-4ed8-aa64-b430619a16ed None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:41.906905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4803fdf4-9d30-4334-8bd1-92f6775f7093 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:41.909425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4803fdf4-9d30-4334-8bd1-92f6775f7093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:42.005794 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 356/712] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:42.007047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803fdf4-9d30-4334-8bd1-92f6775f7093 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:42.037321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c2330-a921-4ed8-aa64-b430619a16ed None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:42.077727 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:42.083172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8f59e-651f-4ef7-8cd6-0409fb6b44da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:42.084187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8f59e-651f-4ef7-8cd6-0409fb6b44da None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:42.084672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8f59e-651f-4ef7-8cd6-0409fb6b44da None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:42.085212 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8f59e-651f-4ef7-8cd6-0409fb6b44da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:42.086387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8f59e-651f-4ef7-8cd6-0409fb6b44da None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=8b7bc20034974816905978dd365993f8) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:42.086963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8f59e-651f-4ef7-8cd6-0409fb6b44da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:42.088765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8f59e-651f-4ef7-8cd6-0409fb6b44da None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:42.103697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 357/713] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:41 2022] GET /identity/v3/role_assignments?user.id=8b7bc20034974816905978dd365993f8&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:42.105096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8f59e-651f-4ef7-8cd6-0409fb6b44da None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:42.124528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803fdf4-9d30-4334-8bd1-92f6775f7093 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:42.130993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:42.173143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a54ee-5fb4-47dc-9f5d-e8038a2649cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:42.173143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a54ee-5fb4-47dc-9f5d-e8038a2649cd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:42.173143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a54ee-5fb4-47dc-9f5d-e8038a2649cd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:42.173143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a54ee-5fb4-47dc-9f5d-e8038a2649cd None admin] PATH_INFO: `/v3/services/network` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:42.173552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a54ee-5fb4-47dc-9f5d-e8038a2649cd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:42.173552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a54ee-5fb4-47dc-9f5d-e8038a2649cd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0d1a54ee-5fb4-47dc-9f5d-e8038a2649cd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:42.191212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:42.192386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:42.193219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 357/714] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:42 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:42.197293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a54ee-5fb4-47dc-9f5d-e8038a2649cd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:42.282262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8f59e-651f-4ef7-8cd6-0409fb6b44da None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:42.320236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c04a8-fb72-468b-a4d9-87c386ae83d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:42.321035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c04a8-fb72-468b-a4d9-87c386ae83d0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:42.321560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c04a8-fb72-468b-a4d9-87c386ae83d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:42.321757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c04a8-fb72-468b-a4d9-87c386ae83d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:42.335600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c04a8-fb72-468b-a4d9-87c386ae83d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:42.337126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c04a8-fb72-468b-a4d9-87c386ae83d0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-392c04a8-fb72-468b-a4d9-87c386ae83d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:42.356830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:42.357859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:42.359447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:42.359447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:42.359447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:42.359447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:42.359447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 358/715] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:42.364138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c04a8-fb72-468b-a4d9-87c386ae83d0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:42.385284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a54ee-5fb4-47dc-9f5d-e8038a2649cd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:42.432013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:42.444535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f176fa8-a305-48d1-8a32-325bb36d0db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:42.445208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f176fa8-a305-48d1-8a32-325bb36d0db8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:42.445490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f176fa8-a305-48d1-8a32-325bb36d0db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:42.445764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f176fa8-a305-48d1-8a32-325bb36d0db8 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:42.446640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f176fa8-a305-48d1-8a32-325bb36d0db8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:42.447080 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f176fa8-a305-48d1-8a32-325bb36d0db8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:42.448769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f176fa8-a305-48d1-8a32-325bb36d0db8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:42.460998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 358/716] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:21:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:42.462959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:42.463808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f176fa8-a305-48d1-8a32-325bb36d0db8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:42.524695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c04a8-fb72-468b-a4d9-87c386ae83d0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:42.598387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f176fa8-a305-48d1-8a32-325bb36d0db8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:42.600513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f5a88-04ae-4e6a-b4b4-4f8080ac6520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:42.601835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5a88-04ae-4e6a-b4b4-4f8080ac6520 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:42.602321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5a88-04ae-4e6a-b4b4-4f8080ac6520 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:42.603112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f5a88-04ae-4e6a-b4b4-4f8080ac6520 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:42.604139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f5a88-04ae-4e6a-b4b4-4f8080ac6520 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:42.604652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f5a88-04ae-4e6a-b4b4-4f8080ac6520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:42.606388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f5a88-04ae-4e6a-b4b4-4f8080ac6520 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:42.633299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020c45c-fab3-4ce5-be75-4b16ce4f987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:42.634018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020c45c-fab3-4ce5-be75-4b16ce4f987b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:42.634308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020c45c-fab3-4ce5-be75-4b16ce4f987b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:42.634588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020c45c-fab3-4ce5-be75-4b16ce4f987b None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:42.635399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020c45c-fab3-4ce5-be75-4b16ce4f987b None admin] RBAC: Adding query filter params (type=network) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:42.635654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020c45c-fab3-4ce5-be75-4b16ce4f987b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:42.637050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020c45c-fab3-4ce5-be75-4b16ce4f987b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:42.637868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 359/717] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:42.642055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 359/718] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:21:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:42.647979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f5a88-04ae-4e6a-b4b4-4f8080ac6520 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:42.823446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f5a88-04ae-4e6a-b4b4-4f8080ac6520 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:42.859888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:42.874187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711b39e-fa0a-47e7-ac1d-49c14a19ce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:42.874873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711b39e-fa0a-47e7-ac1d-49c14a19ce0c None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:42.875152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711b39e-fa0a-47e7-ac1d-49c14a19ce0c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:42.875451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711b39e-fa0a-47e7-ac1d-49c14a19ce0c None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:42.876173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711b39e-fa0a-47e7-ac1d-49c14a19ce0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:42.877605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711b39e-fa0a-47e7-ac1d-49c14a19ce0c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:42.905947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 360/719] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:21:42 2022] POST /identity/v3/users => generated 254 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:21:43.590399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febe177-9fbc-4f37-a557-6880df2cc3d4 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:43.591034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febe177-9fbc-4f37-a557-6880df2cc3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:43.591034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febe177-9fbc-4f37-a557-6880df2cc3d4 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:43.592726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 360/720] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:43.600588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad2c14-4efb-4a72-b86f-139f1f005b98 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:43.601318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad2c14-4efb-4a72-b86f-139f1f005b98 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:43.601318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad2c14-4efb-4a72-b86f-139f1f005b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:43.629152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-51ad2c14-4efb-4a72-b86f-139f1f005b98 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:43.631123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-51ad2c14-4efb-4a72-b86f-139f1f005b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:43.718523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 361/721] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:43.757629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febe177-9fbc-4f37-a557-6880df2cc3d4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:43.844450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:43.906964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febe177-9fbc-4f37-a557-6880df2cc3d4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:43.942943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399b889-0d9e-4c19-b4b6-a193c6a3439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:43.943768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399b889-0d9e-4c19-b4b6-a193c6a3439c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:43.944044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399b889-0d9e-4c19-b4b6-a193c6a3439c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:43.944414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399b889-0d9e-4c19-b4b6-a193c6a3439c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:43.955362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399b889-0d9e-4c19-b4b6-a193c6a3439c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6399b889-0d9e-4c19-b4b6-a193c6a3439c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:43.963139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:43.964171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:43.965224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 361/722] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:43.971904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad2c14-4efb-4a72-b86f-139f1f005b98 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:44.105843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754799-0d55-4608-8102-28a88aafaddd None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:44.106025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754799-0d55-4608-8102-28a88aafaddd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:44.106311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2754799-0d55-4608-8102-28a88aafaddd None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:44.110654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 362/723] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:44.118902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c91f5-bb80-4050-9331-41b5f7048cba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:44.119501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c91f5-bb80-4050-9331-41b5f7048cba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:44.119946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c91f5-bb80-4050-9331-41b5f7048cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:44.140521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad2c14-4efb-4a72-b86f-139f1f005b98 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:44.156407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5e9c91f5-bb80-4050-9331-41b5f7048cba None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:44.157640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5e9c91f5-bb80-4050-9331-41b5f7048cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:44.161194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:44.200809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90393118-446f-4342-84a8-0f6a2c5bfe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:44.201500 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90393118-446f-4342-84a8-0f6a2c5bfe6f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:44.201787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90393118-446f-4342-84a8-0f6a2c5bfe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:44.202054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90393118-446f-4342-84a8-0f6a2c5bfe6f None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:44.202814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90393118-446f-4342-84a8-0f6a2c5bfe6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:44.203129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90393118-446f-4342-84a8-0f6a2c5bfe6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:44.204619 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90393118-446f-4342-84a8-0f6a2c5bfe6f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:44.224104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 362/724] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:44.229383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90393118-446f-4342-84a8-0f6a2c5bfe6f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:44.261075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 363/725] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 02 03:21:44.310848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c91f5-bb80-4050-9331-41b5f7048cba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:44.368661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90393118-446f-4342-84a8-0f6a2c5bfe6f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:44.379663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:44.416192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308edf6f-b4f0-4fdd-8520-e558e1ea39a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:44.417015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308edf6f-b4f0-4fdd-8520-e558e1ea39a2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:44.417338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308edf6f-b4f0-4fdd-8520-e558e1ea39a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:44.417636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308edf6f-b4f0-4fdd-8520-e558e1ea39a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:44.434608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c91f5-bb80-4050-9331-41b5f7048cba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:44.438658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308edf6f-b4f0-4fdd-8520-e558e1ea39a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:44.440172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308edf6f-b4f0-4fdd-8520-e558e1ea39a2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:44.469893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426ded5-c131-4f6e-abdc-8c9ee77ddf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:44.470891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426ded5-c131-4f6e-abdc-8c9ee77ddf0e None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:44.471369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426ded5-c131-4f6e-abdc-8c9ee77ddf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:44.471843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426ded5-c131-4f6e-abdc-8c9ee77ddf0e None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-308edf6f-b4f0-4fdd-8520-e558e1ea39a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:44.472313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:44.473140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:44.474007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:44.474007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:44.474007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:44.474007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:44.474007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 363/726] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:44.475205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426ded5-c131-4f6e-abdc-8c9ee77ddf0e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:44.476791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426ded5-c131-4f6e-abdc-8c9ee77ddf0e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:44.479127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308edf6f-b4f0-4fdd-8520-e558e1ea39a2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:44.502147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 364/727] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:21:44 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:21:44.507794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:44.592932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308edf6f-b4f0-4fdd-8520-e558e1ea39a2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:44.637991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56130e-611c-4a5c-8af9-580306e18546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:44.638651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56130e-611c-4a5c-8af9-580306e18546 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:44.638922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56130e-611c-4a5c-8af9-580306e18546 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:44.639201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56130e-611c-4a5c-8af9-580306e18546 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:44.640059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56130e-611c-4a5c-8af9-580306e18546 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:44.640367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56130e-611c-4a5c-8af9-580306e18546 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:44.641868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56130e-611c-4a5c-8af9-580306e18546 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:44.657216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 364/728] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:44.662314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426ded5-c131-4f6e-abdc-8c9ee77ddf0e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:44.794959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426ded5-c131-4f6e-abdc-8c9ee77ddf0e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:44.802423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:44.837768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ce0dc-b529-4bb8-a00b-1452f39daf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:44.838574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ce0dc-b529-4bb8-a00b-1452f39daf5f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:44.838855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ce0dc-b529-4bb8-a00b-1452f39daf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:44.839135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ce0dc-b529-4bb8-a00b-1452f39daf5f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:44.847186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ce0dc-b529-4bb8-a00b-1452f39daf5f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:44.848674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ce0dc-b529-4bb8-a00b-1452f39daf5f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-306ce0dc-b529-4bb8-a00b-1452f39daf5f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:44.863064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:44.864084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:44.865001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 365/729] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:44.872449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56130e-611c-4a5c-8af9-580306e18546 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:44.942518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9d27b-c495-4230-ba73-f960f54f5ddb None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:44.943133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9d27b-c495-4230-ba73-f960f54f5ddb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:44.943631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9d27b-c495-4230-ba73-f960f54f5ddb None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:44.945527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 366/730] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:44.951942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c044ae2-e565-4473-825e-6d28dfa9702f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:44.952591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c044ae2-e565-4473-825e-6d28dfa9702f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:44.953203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c044ae2-e565-4473-825e-6d28dfa9702f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:44.958834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:44.983326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56130e-611c-4a5c-8af9-580306e18546 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:44.984701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7c044ae2-e565-4473-825e-6d28dfa9702f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:44.986141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7c044ae2-e565-4473-825e-6d28dfa9702f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:45.014258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a993a3f-a968-47f8-904e-36e86b2403cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:45.015184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a993a3f-a968-47f8-904e-36e86b2403cb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.015623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a993a3f-a968-47f8-904e-36e86b2403cb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.016103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a993a3f-a968-47f8-904e-36e86b2403cb None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.017119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a993a3f-a968-47f8-904e-36e86b2403cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:45.017594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a993a3f-a968-47f8-904e-36e86b2403cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:45.019236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a993a3f-a968-47f8-904e-36e86b2403cb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:45.038985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 365/731] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Jul 2 03:21:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:45.049915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a993a3f-a968-47f8-904e-36e86b2403cb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:45.087426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 367/732] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:45.121905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c044ae2-e565-4473-825e-6d28dfa9702f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:45.224794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a993a3f-a968-47f8-904e-36e86b2403cb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:45.229796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:45.236845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c044ae2-e565-4473-825e-6d28dfa9702f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:45.272801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b7dc4-3edd-4fd2-b2b9-c295e6c5d974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:45.274140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b7dc4-3edd-4fd2-b2b9-c295e6c5d974 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.274718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b7dc4-3edd-4fd2-b2b9-c295e6c5d974 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.275143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b7dc4-3edd-4fd2-b2b9-c295e6c5d974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.279489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccd356-f8b2-43f4-b7e5-5221af3183fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:45.280478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccd356-f8b2-43f4-b7e5-5221af3183fa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.283748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccd356-f8b2-43f4-b7e5-5221af3183fa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccd356-f8b2-43f4-b7e5-5221af3183fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.285274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:45.304165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ccd356-f8b2-43f4-b7e5-5221af3183fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-65ccd356-f8b2-43f4-b7e5-5221af3183fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:45.316179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:45.317104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:45.317940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:45.321262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 368/733] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:45.323448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515e6aa-db9b-4e12-adc1-a0543e2de94d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.324353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515e6aa-db9b-4e12-adc1-a0543e2de94d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.325104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515e6aa-db9b-4e12-adc1-a0543e2de94d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.327184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 369/734] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:45.328691 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515e6aa-db9b-4e12-adc1-a0543e2de94d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:45.350884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b7dc4-3edd-4fd2-b2b9-c295e6c5d974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:45.352671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b7dc4-3edd-4fd2-b2b9-c295e6c5d974 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:45.465974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 366/735] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 02 03:21:45.465974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687b311-9a78-45a5-a9a6-5280259dc500 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.465974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687b311-9a78-45a5-a9a6-5280259dc500 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.465974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687b311-9a78-45a5-a9a6-5280259dc500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.471596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515e6aa-db9b-4e12-adc1-a0543e2de94d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:45.484574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3687b311-9a78-45a5-a9a6-5280259dc500 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:45.486594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3687b311-9a78-45a5-a9a6-5280259dc500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:45.507155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228dd57-b20b-4fcc-847d-07a27091b497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:45.507884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228dd57-b20b-4fcc-847d-07a27091b497 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.508153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228dd57-b20b-4fcc-847d-07a27091b497 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.508430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228dd57-b20b-4fcc-847d-07a27091b497 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.509551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228dd57-b20b-4fcc-847d-07a27091b497 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:45.509862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228dd57-b20b-4fcc-847d-07a27091b497 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:45.511281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228dd57-b20b-4fcc-847d-07a27091b497 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:45.520768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 370/736] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:45.521730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228dd57-b20b-4fcc-847d-07a27091b497 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:45.526910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:45.543080 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:45.575793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 367/737] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:45 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 02 03:21:45.577239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687b311-9a78-45a5-a9a6-5280259dc500 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:45.634807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228dd57-b20b-4fcc-847d-07a27091b497 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:45.670344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65782bb8-c261-482b-8970-5b3e89ffce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:45.671986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65782bb8-c261-482b-8970-5b3e89ffce1a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.672475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65782bb8-c261-482b-8970-5b3e89ffce1a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.672951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65782bb8-c261-482b-8970-5b3e89ffce1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.684953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65782bb8-c261-482b-8970-5b3e89ffce1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:45.686884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65782bb8-c261-482b-8970-5b3e89ffce1a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-65782bb8-c261-482b-8970-5b3e89ffce1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:45.705479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:45.706415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:45.707323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:45.707323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:45.707323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:45.707323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:45.708851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687b311-9a78-45a5-a9a6-5280259dc500 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:45.710150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 371/738] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:45.711263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65782bb8-c261-482b-8970-5b3e89ffce1a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:45.756555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2d9e8-079a-44dd-bc79-2000faa5dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:45.757707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2d9e8-079a-44dd-bc79-2000faa5dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.758190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2d9e8-079a-44dd-bc79-2000faa5dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.758719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2d9e8-079a-44dd-bc79-2000faa5dcb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.770050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2d9e8-079a-44dd-bc79-2000faa5dcb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:45.771823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2d9e8-079a-44dd-bc79-2000faa5dcb3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8ae2d9e8-079a-44dd-bc79-2000faa5dcb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:45.790268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:45.791216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:45.792141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:45.792141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:45.792141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:45.792141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:45.793214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:45.795027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 368/739] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:45.796280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2d9e8-079a-44dd-bc79-2000faa5dcb3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:45.821545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:45.832658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65782bb8-c261-482b-8970-5b3e89ffce1a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:45.874943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977676-d622-48f2-8c69-c3c4870d2b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:45.875722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977676-d622-48f2-8c69-c3c4870d2b5e None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.876006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977676-d622-48f2-8c69-c3c4870d2b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.876289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c977676-d622-48f2-8c69-c3c4870d2b5e None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.877183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c977676-d622-48f2-8c69-c3c4870d2b5e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:45.879508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c977676-d622-48f2-8c69-c3c4870d2b5e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:45.898005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 372/740] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:21:45 2022] POST /identity/v3/services => generated 240 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:21:45.899275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977676-d622-48f2-8c69-c3c4870d2b5e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:45.920146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2d9e8-079a-44dd-bc79-2000faa5dcb3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:45.966214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10148639-4185-4901-86d7-97a7d70166fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:45.967032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10148639-4185-4901-86d7-97a7d70166fe None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:45.967332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10148639-4185-4901-86d7-97a7d70166fe None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:45.967836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10148639-4185-4901-86d7-97a7d70166fe None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:45.968324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10148639-4185-4901-86d7-97a7d70166fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:45.968627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10148639-4185-4901-86d7-97a7d70166fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:45.970046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10148639-4185-4901-86d7-97a7d70166fe None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:45.983432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 369/741] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:45.989654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10148639-4185-4901-86d7-97a7d70166fe None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:46.022000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c977676-d622-48f2-8c69-c3c4870d2b5e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:46.057178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:46.067701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a958e56-7d36-4624-bc37-f02260f31fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:46.068754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958e56-7d36-4624-bc37-f02260f31fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.069242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958e56-7d36-4624-bc37-f02260f31fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.069719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a958e56-7d36-4624-bc37-f02260f31fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.070689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a958e56-7d36-4624-bc37-f02260f31fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:46.071277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a958e56-7d36-4624-bc37-f02260f31fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:46.072963 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a958e56-7d36-4624-bc37-f02260f31fe0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:46.079826 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:46.088100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 373/742] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:46.089959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f0ddc-008e-47c5-8064-6bad0939ac8c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.090449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f0ddc-008e-47c5-8064-6bad0939ac8c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.090940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f0ddc-008e-47c5-8064-6bad0939ac8c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.093029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 374/743] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:46.094053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f0ddc-008e-47c5-8064-6bad0939ac8c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:46.107997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10148639-4185-4901-86d7-97a7d70166fe None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:46.157913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43c02-7f16-486c-8486-fe5064700568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:46.158752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43c02-7f16-486c-8486-fe5064700568 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.159034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43c02-7f16-486c-8486-fe5064700568 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.159318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43c02-7f16-486c-8486-fe5064700568 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.170909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43c02-7f16-486c-8486-fe5064700568 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:46.173075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f43c02-7f16-486c-8486-fe5064700568 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-59f43c02-7f16-486c-8486-fe5064700568 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:46.188917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:46.189862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:46.190755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:46.190755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:46.190755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:46.190755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:46.190755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:46.190755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:46.190755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:46.190755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:46.190755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 370/744] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:46.191817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b18ac7-7803-45b1-945c-90a70267b3d4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.194099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b18ac7-7803-45b1-945c-90a70267b3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.194099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b18ac7-7803-45b1-945c-90a70267b3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.209517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-60b18ac7-7803-45b1-945c-90a70267b3d4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:46.210711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-60b18ac7-7803-45b1-945c-90a70267b3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:46.213613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f0ddc-008e-47c5-8064-6bad0939ac8c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:46.257327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9d9cf-0582-49fd-9024-ad1fb81844ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:46.258151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d9cf-0582-49fd-9024-ad1fb81844ea None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.258435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d9cf-0582-49fd-9024-ad1fb81844ea None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.259132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d9cf-0582-49fd-9024-ad1fb81844ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.268454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9d9cf-0582-49fd-9024-ad1fb81844ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:46.270421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9d9cf-0582-49fd-9024-ad1fb81844ea None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:46.281226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 371/745] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:46.282222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b18ac7-7803-45b1-945c-90a70267b3d4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b9e9d9cf-0582-49fd-9024-ad1fb81844ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:46.288047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:46.289023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:46.290041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 375/746] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Jul 2 03:21:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:46.296610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9d9cf-0582-49fd-9024-ad1fb81844ea None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:46.354358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:46.358805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:46.395260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b18ac7-7803-45b1-945c-90a70267b3d4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:46.421242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9d9cf-0582-49fd-9024-ad1fb81844ea None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:46.432827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20d8c2-1ce2-48ba-b77c-f3de1e2750b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:46.433655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20d8c2-1ce2-48ba-b77c-f3de1e2750b5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.434041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20d8c2-1ce2-48ba-b77c-f3de1e2750b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.434438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20d8c2-1ce2-48ba-b77c-f3de1e2750b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.435360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20d8c2-1ce2-48ba-b77c-f3de1e2750b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:46.435770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20d8c2-1ce2-48ba-b77c-f3de1e2750b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:46.437425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20d8c2-1ce2-48ba-b77c-f3de1e2750b5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:46.455625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 372/747] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:46.455625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20d8c2-1ce2-48ba-b77c-f3de1e2750b5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:46.478612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ef7b1-2f35-4c30-8db1-7870ae959a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:46.479963 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ef7b1-2f35-4c30-8db1-7870ae959a1d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.480848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ef7b1-2f35-4c30-8db1-7870ae959a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.481580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ef7b1-2f35-4c30-8db1-7870ae959a1d None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.482615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ef7b1-2f35-4c30-8db1-7870ae959a1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:46.483425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ef7b1-2f35-4c30-8db1-7870ae959a1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:46.485772 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ef7b1-2f35-4c30-8db1-7870ae959a1d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:46.510203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 376/748] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Jul 2 03:21:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:46.511472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ef7b1-2f35-4c30-8db1-7870ae959a1d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:46.565702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20d8c2-1ce2-48ba-b77c-f3de1e2750b5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:46.594850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:46.608436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836e4d4-4afc-4679-af5d-ca86c55cb618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:46.609503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836e4d4-4afc-4679-af5d-ca86c55cb618 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.609985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836e4d4-4afc-4679-af5d-ca86c55cb618 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.610925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836e4d4-4afc-4679-af5d-ca86c55cb618 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.625125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836e4d4-4afc-4679-af5d-ca86c55cb618 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-4836e4d4-4afc-4679-af5d-ca86c55cb618 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:46.636305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:46.637253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:46.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:46.638687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ef7b1-2f35-4c30-8db1-7870ae959a1d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:46.646611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 373/749] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:46.647703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836e4d4-4afc-4679-af5d-ca86c55cb618 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:46.648949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:46.677557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f070b-2143-4a47-9b1f-2be564e414b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:46.678383 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f070b-2143-4a47-9b1f-2be564e414b1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.678766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f070b-2143-4a47-9b1f-2be564e414b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.679141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f070b-2143-4a47-9b1f-2be564e414b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.680107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f070b-2143-4a47-9b1f-2be564e414b1 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=f37a95ba40774cfd89e6f397afa301a9) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:46.680573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f070b-2143-4a47-9b1f-2be564e414b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:46.682645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f070b-2143-4a47-9b1f-2be564e414b1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:46.695319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 377/750] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:46 2022] GET /identity/v3/role_assignments?user.id=f37a95ba40774cfd89e6f397afa301a9&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:46.695319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f070b-2143-4a47-9b1f-2be564e414b1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:46.843838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836e4d4-4afc-4679-af5d-ca86c55cb618 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:46.896083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f070b-2143-4a47-9b1f-2be564e414b1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:46.906986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e27e37-8b69-4029-ad96-930ddeb3c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:46.908130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e27e37-8b69-4029-ad96-930ddeb3c301 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.908604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e27e37-8b69-4029-ad96-930ddeb3c301 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.909063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e27e37-8b69-4029-ad96-930ddeb3c301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.926720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e27e37-8b69-4029-ad96-930ddeb3c301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:46.929103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e27e37-8b69-4029-ad96-930ddeb3c301 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-07e27e37-8b69-4029-ad96-930ddeb3c301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:46.948901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:46.949821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:46.950724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:46.950724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:46.950724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:46.950724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:46.953178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 374/751] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:46.958466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e27e37-8b69-4029-ad96-930ddeb3c301 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:46.973802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd77c70-2046-433c-bec6-a5033f35b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:46.976505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd77c70-2046-433c-bec6-a5033f35b266 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:46.976999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd77c70-2046-433c-bec6-a5033f35b266 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:46.979129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd77c70-2046-433c-bec6-a5033f35b266 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:46.980142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd77c70-2046-433c-bec6-a5033f35b266 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:46.980655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd77c70-2046-433c-bec6-a5033f35b266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:46.982228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd77c70-2046-433c-bec6-a5033f35b266 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.009148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 378/752] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:47.022774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110bab3-409f-4301-97cc-f1854264c049 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.023429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110bab3-409f-4301-97cc-f1854264c049 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.023924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110bab3-409f-4301-97cc-f1854264c049 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.025789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 379/753] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:21:47.026682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110bab3-409f-4301-97cc-f1854264c049 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:47.033556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:47.067489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:47.096872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e27e37-8b69-4029-ad96-930ddeb3c301 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.142639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e65fb-1d86-4d55-8c8a-0397ffb255c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.143258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110bab3-409f-4301-97cc-f1854264c049 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.143405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e65fb-1d86-4d55-8c8a-0397ffb255c1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.143601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e65fb-1d86-4d55-8c8a-0397ffb255c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.143878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e65fb-1d86-4d55-8c8a-0397ffb255c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.144605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e65fb-1d86-4d55-8c8a-0397ffb255c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:47.144912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e65fb-1d86-4d55-8c8a-0397ffb255c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.146335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e65fb-1d86-4d55-8c8a-0397ffb255c1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.164618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 375/754] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:47.166080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159c9c6-b69c-4412-8698-e6a0a7dd15c1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.166367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159c9c6-b69c-4412-8698-e6a0a7dd15c1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.166643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159c9c6-b69c-4412-8698-e6a0a7dd15c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.183835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b159c9c6-b69c-4412-8698-e6a0a7dd15c1 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:47.184147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c9338-8cc8-4597-bb3f-954302d8fbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.185026 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b159c9c6-b69c-4412-8698-e6a0a7dd15c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:47.185398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c9338-8cc8-4597-bb3f-954302d8fbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.185866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c9338-8cc8-4597-bb3f-954302d8fbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.186323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c9338-8cc8-4597-bb3f-954302d8fbb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.198005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c9338-8cc8-4597-bb3f-954302d8fbb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.200056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c9338-8cc8-4597-bb3f-954302d8fbb0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-cb6c9338-8cc8-4597-bb3f-954302d8fbb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:47.218592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:47.219543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:47.220420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:47.220420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:47.220420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:47.220420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:47.222734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 380/755] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:47.223835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c9338-8cc8-4597-bb3f-954302d8fbb0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:47.259560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 376/756] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:47.260471 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159c9c6-b69c-4412-8698-e6a0a7dd15c1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:47.307519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:47.323668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:47.335773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c9338-8cc8-4597-bb3f-954302d8fbb0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.372068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e565a20-d321-44c9-92d7-c2d3c459e58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.373169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e565a20-d321-44c9-92d7-c2d3c459e58b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.373642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e565a20-d321-44c9-92d7-c2d3c459e58b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.374102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e565a20-d321-44c9-92d7-c2d3c459e58b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.381009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e565a20-d321-44c9-92d7-c2d3c459e58b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.382792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e565a20-d321-44c9-92d7-c2d3c459e58b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.397180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159c9c6-b69c-4412-8698-e6a0a7dd15c1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1e565a20-d321-44c9-92d7-c2d3c459e58b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:47.398015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:47.398856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:47.399929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:47.399929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:47.399929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:47.399929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:47.399929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:47.399929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:47.399929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:47.399929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:47.402501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 381/757] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:47.403540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e565a20-d321-44c9-92d7-c2d3c459e58b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:47.439092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f532d-7cfb-488e-b3ce-d7d59e74c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.440301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f532d-7cfb-488e-b3ce-d7d59e74c6ee None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.440776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f532d-7cfb-488e-b3ce-d7d59e74c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.441285 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f532d-7cfb-488e-b3ce-d7d59e74c6ee None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.442256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f532d-7cfb-488e-b3ce-d7d59e74c6ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:47.442801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f532d-7cfb-488e-b3ce-d7d59e74c6ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.444622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f532d-7cfb-488e-b3ce-d7d59e74c6ee None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.460361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 377/758] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:47.461741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f532d-7cfb-488e-b3ce-d7d59e74c6ee None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:47.541403 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e565a20-d321-44c9-92d7-c2d3c459e58b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.562777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:47.579781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f532d-7cfb-488e-b3ce-d7d59e74c6ee None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.591549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cd9b6-9fc1-40b9-98e9-9c370ad9df22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.592783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cd9b6-9fc1-40b9-98e9-9c370ad9df22 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.593073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cd9b6-9fc1-40b9-98e9-9c370ad9df22 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.593355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cd9b6-9fc1-40b9-98e9-9c370ad9df22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.594309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cd9b6-9fc1-40b9-98e9-9c370ad9df22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.596046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cd9b6-9fc1-40b9-98e9-9c370ad9df22 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c18cd9b6-9fc1-40b9-98e9-9c370ad9df22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:47.612779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:47.613717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:47.614538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 382/759] 10.222.0.42 () {56 vars in 1129 bytes} [Sat Jul 2 03:21:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:47.615402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cd9b6-9fc1-40b9-98e9-9c370ad9df22 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:47.621465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4864bf-bbb3-4298-9eac-5cf144a76f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.622489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4864bf-bbb3-4298-9eac-5cf144a76f3e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.623123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4864bf-bbb3-4298-9eac-5cf144a76f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.623606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4864bf-bbb3-4298-9eac-5cf144a76f3e None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.624595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4864bf-bbb3-4298-9eac-5cf144a76f3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:47.625136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4864bf-bbb3-4298-9eac-5cf144a76f3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.626842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4864bf-bbb3-4298-9eac-5cf144a76f3e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.630249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:47.643062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 378/760] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:47.644255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4864bf-bbb3-4298-9eac-5cf144a76f3e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:47.750169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cd9b6-9fc1-40b9-98e9-9c370ad9df22 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.756975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4864bf-bbb3-4298-9eac-5cf144a76f3e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.790094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c799799-0eee-4535-a2e5-3b8583f2e720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.791009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c799799-0eee-4535-a2e5-3b8583f2e720 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.791316 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c799799-0eee-4535-a2e5-3b8583f2e720 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.791669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c799799-0eee-4535-a2e5-3b8583f2e720 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.797213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a57618-d4bf-4e89-a17f-57524a9120ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.798290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a57618-d4bf-4e89-a17f-57524a9120ec None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.798784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a57618-d4bf-4e89-a17f-57524a9120ec None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.799242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a57618-d4bf-4e89-a17f-57524a9120ec None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.799788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c799799-0eee-4535-a2e5-3b8583f2e720 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.800358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a57618-d4bf-4e89-a17f-57524a9120ec None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:47.800861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a57618-d4bf-4e89-a17f-57524a9120ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.801304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c799799-0eee-4535-a2e5-3b8583f2e720 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.802993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a57618-d4bf-4e89-a17f-57524a9120ec None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.809005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 379/761] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Jul 2 03:21:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:47.810042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a57618-d4bf-4e89-a17f-57524a9120ec None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2c799799-0eee-4535-a2e5-3b8583f2e720 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:47.817730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:47.819510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:21:47.821355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:47.823764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 383/762] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:21:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:47.824738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c799799-0eee-4535-a2e5-3b8583f2e720 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:47.834294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:47.904703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:47.937454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a57618-d4bf-4e89-a17f-57524a9120ec None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.939319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c799799-0eee-4535-a2e5-3b8583f2e720 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:47.974842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687aaaf2-2da7-422a-ba11-c21de412fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.975276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297ab5-5018-419c-8f9a-8eb68e1daca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:47.975985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297ab5-5018-419c-8f9a-8eb68e1daca0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.976272 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297ab5-5018-419c-8f9a-8eb68e1daca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.976554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297ab5-5018-419c-8f9a-8eb68e1daca0 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.977112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687aaaf2-2da7-422a-ba11-c21de412fd31 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:47.977330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd297ab5-5018-419c-8f9a-8eb68e1daca0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:47.977786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd297ab5-5018-419c-8f9a-8eb68e1daca0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.977990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687aaaf2-2da7-422a-ba11-c21de412fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:47.978582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687aaaf2-2da7-422a-ba11-c21de412fd31 None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/8b7bc20034974816905978dd365993f8/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:47.979186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd297ab5-5018-419c-8f9a-8eb68e1daca0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.981090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687aaaf2-2da7-422a-ba11-c21de412fd31 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=8b7bc20034974816905978dd365993f8, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:47.982762 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687aaaf2-2da7-422a-ba11-c21de412fd31 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:47.984852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 384/763] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Jul 2 03:21:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:47.985876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297ab5-5018-419c-8f9a-8eb68e1daca0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:48.000030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 380/764] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:21:47 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/8b7bc20034974816905978dd365993f8/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:48.001331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687aaaf2-2da7-422a-ba11-c21de412fd31 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:48.081264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:48.116983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687aaaf2-2da7-422a-ba11-c21de412fd31 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:48.132711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297ab5-5018-419c-8f9a-8eb68e1daca0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:48.166880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646995ad-49d7-4541-be5d-2ef2e1af129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:48.167540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646995ad-49d7-4541-be5d-2ef2e1af129d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.167955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646995ad-49d7-4541-be5d-2ef2e1af129d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.168232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646995ad-49d7-4541-be5d-2ef2e1af129d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.169107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646995ad-49d7-4541-be5d-2ef2e1af129d None admin] RBAC: Adding query filter params (interface=public, service_id=782db742c3734abd8b9c9a8a98cd4747, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:48.169419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646995ad-49d7-4541-be5d-2ef2e1af129d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:48.170940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646995ad-49d7-4541-be5d-2ef2e1af129d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:48.180366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 381/765] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Jul 2 03:21:48 2022] GET /identity/v3/endpoints?service_id=782db742c3734abd8b9c9a8a98cd4747&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:48.182209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2cdcc-cf8e-4581-aa97-7dbff35fd0ee None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.182492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2cdcc-cf8e-4581-aa97-7dbff35fd0ee None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.182767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd2cdcc-cf8e-4581-aa97-7dbff35fd0ee None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.183878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04110c6e-8188-4570-a87d-cb4602dabe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:48.189667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 382/766] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:48.189667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2cdcc-cf8e-4581-aa97-7dbff35fd0ee None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:48.192662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04110c6e-8188-4570-a87d-cb4602dabe73 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.193387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04110c6e-8188-4570-a87d-cb4602dabe73 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.194062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04110c6e-8188-4570-a87d-cb4602dabe73 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.195819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04110c6e-8188-4570-a87d-cb4602dabe73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:48.196713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04110c6e-8188-4570-a87d-cb4602dabe73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:48.200491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04110c6e-8188-4570-a87d-cb4602dabe73 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:48.205043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:48.232857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 385/767] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Jul 2 03:21:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:48.235214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8b012-3728-4230-8e57-83060d96cd1c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.235534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8b012-3728-4230-8e57-83060d96cd1c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.235812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8b012-3728-4230-8e57-83060d96cd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.254839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-59d8b012-3728-4230-8e57-83060d96cd1c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:48.256453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-59d8b012-3728-4230-8e57-83060d96cd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:48.328461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 386/768] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:48.335099 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8b012-3728-4230-8e57-83060d96cd1c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:48.337149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd2cdcc-cf8e-4581-aa97-7dbff35fd0ee None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:48.390481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cef76d-150b-44f9-9ca4-bfac6f922d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:48.391814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cef76d-150b-44f9-9ca4-bfac6f922d9b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.392239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cef76d-150b-44f9-9ca4-bfac6f922d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.392996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cef76d-150b-44f9-9ca4-bfac6f922d9b None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.393821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cef76d-150b-44f9-9ca4-bfac6f922d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:48.396129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cef76d-150b-44f9-9ca4-bfac6f922d9b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:48.399054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:48.405792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 383/769] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Jul 2 03:21:48 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:48.407385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cef76d-150b-44f9-9ca4-bfac6f922d9b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:48.490959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8b012-3728-4230-8e57-83060d96cd1c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:48.540367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:48.542314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cef76d-150b-44f9-9ca4-bfac6f922d9b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:48.584372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec897bc-ef7f-4e2e-a685-7676165063e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:48.584937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec897bc-ef7f-4e2e-a685-7676165063e4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.585219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec897bc-ef7f-4e2e-a685-7676165063e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.585494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec897bc-ef7f-4e2e-a685-7676165063e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.594010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401d145-3101-419d-80c8-ea2f8169b947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:48.595563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401d145-3101-419d-80c8-ea2f8169b947 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.596099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401d145-3101-419d-80c8-ea2f8169b947 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.596528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401d145-3101-419d-80c8-ea2f8169b947 None admin] PATH_INFO: `/v3/services/network` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.597485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401d145-3101-419d-80c8-ea2f8169b947 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:48.600069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401d145-3101-419d-80c8-ea2f8169b947 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2401d145-3101-419d-80c8-ea2f8169b947 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:48.624588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:48.625810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 02 03:21:48.626749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:48.629197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 384/770] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:48 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:48.635458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401d145-3101-419d-80c8-ea2f8169b947 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:48.673126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec897bc-ef7f-4e2e-a685-7676165063e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:48.675059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec897bc-ef7f-4e2e-a685-7676165063e4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:48.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:48.747662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401d145-3101-419d-80c8-ea2f8169b947 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:48.777325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 387/771] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:21:48.782533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec897bc-ef7f-4e2e-a685-7676165063e4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:48.785106 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea956b-525e-4eb8-9e33-161c75c1d9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:48.786028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea956b-525e-4eb8-9e33-161c75c1d9ac None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.786447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea956b-525e-4eb8-9e33-161c75c1d9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.786856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cea956b-525e-4eb8-9e33-161c75c1d9ac None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.787740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea956b-525e-4eb8-9e33-161c75c1d9ac None admin] RBAC: Adding query filter params (name=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:48.788386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea956b-525e-4eb8-9e33-161c75c1d9ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:48.790056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cea956b-525e-4eb8-9e33-161c75c1d9ac None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:48.795995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 385/772] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:21:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:48.801694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea956b-525e-4eb8-9e33-161c75c1d9ac None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:48.809247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:48.887211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec897bc-ef7f-4e2e-a685-7676165063e4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:48.923260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9b70b-bbcc-4793-a4be-a2bbb1720802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:48.924105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9b70b-bbcc-4793-a4be-a2bbb1720802 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.924390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9b70b-bbcc-4793-a4be-a2bbb1720802 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.924667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9b70b-bbcc-4793-a4be-a2bbb1720802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.925555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cea956b-525e-4eb8-9e33-161c75c1d9ac None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:48.935692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9b70b-bbcc-4793-a4be-a2bbb1720802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:48.937171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9b70b-bbcc-4793-a4be-a2bbb1720802 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:48.954652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f9b9b70b-bbcc-4793-a4be-a2bbb1720802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:48.955311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:48.956339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:48.957313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:48.957313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:48.957313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:48.957313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:48.957313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 388/773] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:48.962696 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9b70b-bbcc-4793-a4be-a2bbb1720802 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:48.967056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148604-c26f-4e04-9a33-59d1d2be5e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:48.967930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148604-c26f-4e04-9a33-59d1d2be5e44 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:48.968347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148604-c26f-4e04-9a33-59d1d2be5e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:48.968775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52148604-c26f-4e04-9a33-59d1d2be5e44 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:48.969609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52148604-c26f-4e04-9a33-59d1d2be5e44 None admin] RBAC: Adding query filter params (type=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:48.970049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52148604-c26f-4e04-9a33-59d1d2be5e44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:48.971692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52148604-c26f-4e04-9a33-59d1d2be5e44 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:48.978863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 386/774] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:21:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:48.984719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148604-c26f-4e04-9a33-59d1d2be5e44 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:49.038146 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:49.064065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9b70b-bbcc-4793-a4be-a2bbb1720802 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:49.094826 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52148604-c26f-4e04-9a33-59d1d2be5e44 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:49.099872 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fe63a-e467-43fd-8956-90ea50b3b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:49.100542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fe63a-e467-43fd-8956-90ea50b3b084 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.100820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fe63a-e467-43fd-8956-90ea50b3b084 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.101095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fe63a-e467-43fd-8956-90ea50b3b084 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.101818 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fe63a-e467-43fd-8956-90ea50b3b084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:49.102126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fe63a-e467-43fd-8956-90ea50b3b084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:49.103775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fe63a-e467-43fd-8956-90ea50b3b084 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:49.117822 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 389/775] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:49.119205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a6f57-f0d8-4b6f-8a32-b33532bbc822 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.123978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a6f57-f0d8-4b6f-8a32-b33532bbc822 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.124282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a6f57-f0d8-4b6f-8a32-b33532bbc822 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.125882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 390/776] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:49.126602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a6f57-f0d8-4b6f-8a32-b33532bbc822 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:49.136135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b835c8-7e45-4326-b055-18448eb9c954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:49.137034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b835c8-7e45-4326-b055-18448eb9c954 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.137455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b835c8-7e45-4326-b055-18448eb9c954 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.137895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b835c8-7e45-4326-b055-18448eb9c954 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.138870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b835c8-7e45-4326-b055-18448eb9c954 None admin] RBAC: Adding query filter params (interface=public, service_id=ea9b94ebca5947eaa7290b4cbd40eeed, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:49.139346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b835c8-7e45-4326-b055-18448eb9c954 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:49.141020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b835c8-7e45-4326-b055-18448eb9c954 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:49.151573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 387/777] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Jul 2 03:21:48 2022] GET /identity/v3/endpoints?service_id=ea9b94ebca5947eaa7290b4cbd40eeed&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:49.153332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3333a-e698-42b7-bea8-e9467f30bd5e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.153760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3333a-e698-42b7-bea8-e9467f30bd5e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.154176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3333a-e698-42b7-bea8-e9467f30bd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.171939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5af3333a-e698-42b7-bea8-e9467f30bd5e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:49.173353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5af3333a-e698-42b7-bea8-e9467f30bd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:49.230356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:49.239605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a6f57-f0d8-4b6f-8a32-b33532bbc822 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:49.259696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 388/778] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:49.260468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3333a-e698-42b7-bea8-e9467f30bd5e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:49.292800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f501e-85b4-4f6b-82ba-563ba6453e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:49.293947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f501e-85b4-4f6b-82ba-563ba6453e24 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.294621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f501e-85b4-4f6b-82ba-563ba6453e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.295036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f501e-85b4-4f6b-82ba-563ba6453e24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.298135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:49.307674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f501e-85b4-4f6b-82ba-563ba6453e24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:49.309161 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f501e-85b4-4f6b-82ba-563ba6453e24 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e54f501e-85b4-4f6b-82ba-563ba6453e24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:49.331552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:49.332500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:49.333400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:49.333400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:49.333400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:49.333400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:49.333400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:49.333400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:49.333400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:49.333400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:49.336137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 391/779] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:49.337704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f501e-85b4-4f6b-82ba-563ba6453e24 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:49.402652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3333a-e698-42b7-bea8-e9467f30bd5e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:49.445776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f16de1-85b5-4dc0-95bc-8794f377e190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:49.446635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f16de1-85b5-4dc0-95bc-8794f377e190 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.447108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f16de1-85b5-4dc0-95bc-8794f377e190 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.447474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f501e-85b4-4f6b-82ba-563ba6453e24 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:49.447607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f16de1-85b5-4dc0-95bc-8794f377e190 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.448392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f16de1-85b5-4dc0-95bc-8794f377e190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:49.449862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f16de1-85b5-4dc0-95bc-8794f377e190 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:49.455742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 389/780] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Jul 2 03:21:49 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:49.456655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f16de1-85b5-4dc0-95bc-8794f377e190 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:49.512078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b26b2d-0dbc-4d5f-bbf4-656269493350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:49.512935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b26b2d-0dbc-4d5f-bbf4-656269493350 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.513218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b26b2d-0dbc-4d5f-bbf4-656269493350 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.513493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b26b2d-0dbc-4d5f-bbf4-656269493350 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.534969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b26b2d-0dbc-4d5f-bbf4-656269493350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-33b26b2d-0dbc-4d5f-bbf4-656269493350 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:49.544015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:49.544999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:49.545825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:49.551333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 392/781] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:49.556981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b26b2d-0dbc-4d5f-bbf4-656269493350 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:49.558384 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:49.624040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:49.667577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f16de1-85b5-4dc0-95bc-8794f377e190 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:49.667789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b26b2d-0dbc-4d5f-bbf4-656269493350 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:49.705531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11320903-f3d3-4ef4-8080-74587253af41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:49.706044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae94cc4-b203-4026-90a1-9fcd68117f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:49.706199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11320903-f3d3-4ef4-8080-74587253af41 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.706487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11320903-f3d3-4ef4-8080-74587253af41 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.706763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11320903-f3d3-4ef4-8080-74587253af41 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.707284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae94cc4-b203-4026-90a1-9fcd68117f19 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.707580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11320903-f3d3-4ef4-8080-74587253af41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:49.707870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae94cc4-b203-4026-90a1-9fcd68117f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.707989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11320903-f3d3-4ef4-8080-74587253af41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:49.708386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae94cc4-b203-4026-90a1-9fcd68117f19 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.709267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae94cc4-b203-4026-90a1-9fcd68117f19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:49.709374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11320903-f3d3-4ef4-8080-74587253af41 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:49.709795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae94cc4-b203-4026-90a1-9fcd68117f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:49.711519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae94cc4-b203-4026-90a1-9fcd68117f19 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:49.721809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 393/782] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:49.724773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 390/783] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:49.727342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae94cc4-b203-4026-90a1-9fcd68117f19 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:49.732477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11320903-f3d3-4ef4-8080-74587253af41 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:49.851214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11320903-f3d3-4ef4-8080-74587253af41 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:49.861429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae94cc4-b203-4026-90a1-9fcd68117f19 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:49.863741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:49.878876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:49.892171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531ddc-b0d5-48da-a1e6-2ba25407c398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:49.892925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531ddc-b0d5-48da-a1e6-2ba25407c398 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.893210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531ddc-b0d5-48da-a1e6-2ba25407c398 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.893482 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f531ddc-b0d5-48da-a1e6-2ba25407c398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.894310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f531ddc-b0d5-48da-a1e6-2ba25407c398 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=2fcefa2d73a84415875357ef03e2c8da) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:49.894626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f531ddc-b0d5-48da-a1e6-2ba25407c398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:49.896084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f531ddc-b0d5-48da-a1e6-2ba25407c398 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:49.898140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347fb8c-0996-4b46-802c-92fd79350a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:49.899205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347fb8c-0996-4b46-802c-92fd79350a47 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:49.899766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347fb8c-0996-4b46-802c-92fd79350a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:49.900250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347fb8c-0996-4b46-802c-92fd79350a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:49.902554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 391/784] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:49 2022] GET /identity/v3/role_assignments?user.id=2fcefa2d73a84415875357ef03e2c8da&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:49.912676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347fb8c-0996-4b46-802c-92fd79350a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:49.914571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9347fb8c-0996-4b46-802c-92fd79350a47 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-9347fb8c-0996-4b46-802c-92fd79350a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:49.951224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:49.952233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:49.953129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:49.953129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:49.953129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:49.953129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:49.956702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 394/785] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:49.962101 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531ddc-b0d5-48da-a1e6-2ba25407c398 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:50.085246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f531ddc-b0d5-48da-a1e6-2ba25407c398 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:50.123113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f15e8-92f1-4eac-9a5f-e137158c9de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:50.123818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f15e8-92f1-4eac-9a5f-e137158c9de9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:50.124095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f15e8-92f1-4eac-9a5f-e137158c9de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:50.124364 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f15e8-92f1-4eac-9a5f-e137158c9de9 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:50.125090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f15e8-92f1-4eac-9a5f-e137158c9de9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:50.125391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f15e8-92f1-4eac-9a5f-e137158c9de9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:50.129045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f15e8-92f1-4eac-9a5f-e137158c9de9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:50.140860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 392/786] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:50.145973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347fb8c-0996-4b46-802c-92fd79350a47 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:50.285389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347fb8c-0996-4b46-802c-92fd79350a47 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:50.320167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f93022-0921-4e04-90f5-990c8026f1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:50.320965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f93022-0921-4e04-90f5-990c8026f1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:50.321271 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f93022-0921-4e04-90f5-990c8026f1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:50.321549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f93022-0921-4e04-90f5-990c8026f1b9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:50.329716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f93022-0921-4e04-90f5-990c8026f1b9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:50.331195 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f93022-0921-4e04-90f5-990c8026f1b9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-11f93022-0921-4e04-90f5-990c8026f1b9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:50.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:50.346434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:50.347391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 395/787] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:50.352529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f15e8-92f1-4eac-9a5f-e137158c9de9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:50.362154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e5c3a-78cd-42a4-bde7-3cb626bb01de None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:50.362494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e5c3a-78cd-42a4-bde7-3cb626bb01de None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:50.362725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e5c3a-78cd-42a4-bde7-3cb626bb01de None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:50.364384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 396/788] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:21:50.373698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82130151-54b5-4dd9-bb4a-e6e179848d42 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:50.373981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82130151-54b5-4dd9-bb4a-e6e179848d42 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:50.374262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82130151-54b5-4dd9-bb4a-e6e179848d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:50.404407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-82130151-54b5-4dd9-bb4a-e6e179848d42 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:50.405597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-82130151-54b5-4dd9-bb4a-e6e179848d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:50.427636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:50.456236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:50.513973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 397/789] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:50.537300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f15e8-92f1-4eac-9a5f-e137158c9de9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:50.546607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82130151-54b5-4dd9-bb4a-e6e179848d42 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:50.608567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5628f-b561-4bc4-b731-50af19f4641e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:50.609253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5628f-b561-4bc4-b731-50af19f4641e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:50.609533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5628f-b561-4bc4-b731-50af19f4641e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:50.609802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5628f-b561-4bc4-b731-50af19f4641e None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:50.610646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5628f-b561-4bc4-b731-50af19f4641e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:50.610954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5628f-b561-4bc4-b731-50af19f4641e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:50.612424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5628f-b561-4bc4-b731-50af19f4641e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:50.646140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 393/790] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Jul 2 03:21:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:50.653263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5628f-b561-4bc4-b731-50af19f4641e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:50.754833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:50.759634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82130151-54b5-4dd9-bb4a-e6e179848d42 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:50.771648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5628f-b561-4bc4-b731-50af19f4641e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:50.827792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168eb86f-9759-4da8-8289-9ef36511e7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:50.828589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168eb86f-9759-4da8-8289-9ef36511e7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:50.828866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168eb86f-9759-4da8-8289-9ef36511e7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:50.829139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168eb86f-9759-4da8-8289-9ef36511e7b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:50.844930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491b65c-1f17-4f14-bd15-c19d377f3eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:50.845751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491b65c-1f17-4f14-bd15-c19d377f3eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:50.846025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491b65c-1f17-4f14-bd15-c19d377f3eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:50.846295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491b65c-1f17-4f14-bd15-c19d377f3eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:50.856246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168eb86f-9759-4da8-8289-9ef36511e7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:50.868237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491b65c-1f17-4f14-bd15-c19d377f3eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:50.869601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491b65c-1f17-4f14-bd15-c19d377f3eb8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-168eb86f-9759-4da8-8289-9ef36511e7b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:50.872606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:50.873892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:50.874724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:50.880986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 398/791] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:50.881870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168eb86f-9759-4da8-8289-9ef36511e7b5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0491b65c-1f17-4f14-bd15-c19d377f3eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:50.903601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:50.905763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:50.907813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:50.907813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:50.907813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:50.907813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:50.908210 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:50.911442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 394/792] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:50.912323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491b65c-1f17-4f14-bd15-c19d377f3eb8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:51.060795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168eb86f-9759-4da8-8289-9ef36511e7b5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:51.081007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491b65c-1f17-4f14-bd15-c19d377f3eb8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:51.129015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e657b-6c73-49e7-848f-6fdaa21cd2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:51.129015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e657b-6c73-49e7-848f-6fdaa21cd2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.129015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e657b-6c73-49e7-848f-6fdaa21cd2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.129015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e657b-6c73-49e7-848f-6fdaa21cd2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.129383 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e657b-6c73-49e7-848f-6fdaa21cd2b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:51.129383 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e657b-6c73-49e7-848f-6fdaa21cd2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:51.130735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e657b-6c73-49e7-848f-6fdaa21cd2b1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:51.150088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce332-fbae-4ccf-9bfc-dc310688c0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:51.150749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce332-fbae-4ccf-9bfc-dc310688c0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.151023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce332-fbae-4ccf-9bfc-dc310688c0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.151326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce332-fbae-4ccf-9bfc-dc310688c0a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.152033 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ce332-fbae-4ccf-9bfc-dc310688c0a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:51.152335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ce332-fbae-4ccf-9bfc-dc310688c0a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:51.153755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ce332-fbae-4ccf-9bfc-dc310688c0a5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:51.155418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 399/793] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:51.156300 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e657b-6c73-49e7-848f-6fdaa21cd2b1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:51.185616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 395/794] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:51.201310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce332-fbae-4ccf-9bfc-dc310688c0a5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:51.223713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:51.269810 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:51.291649 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e657b-6c73-49e7-848f-6fdaa21cd2b1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:51.332012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896a1bd-7a14-4eb6-a1b0-4463eedb4717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:51.333105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896a1bd-7a14-4eb6-a1b0-4463eedb4717 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.333528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896a1bd-7a14-4eb6-a1b0-4463eedb4717 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.333941 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896a1bd-7a14-4eb6-a1b0-4463eedb4717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.335508 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce332-fbae-4ccf-9bfc-dc310688c0a5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:51.345938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896a1bd-7a14-4eb6-a1b0-4463eedb4717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:51.347780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896a1bd-7a14-4eb6-a1b0-4463eedb4717 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f896a1bd-7a14-4eb6-a1b0-4463eedb4717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:51.366510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:51.367471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:51.368387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:51.368387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:51.368387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:51.368387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:51.375044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074d44d-c192-492a-9164-1eeaa65d7ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:51.375044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074d44d-c192-492a-9164-1eeaa65d7ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.375044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074d44d-c192-492a-9164-1eeaa65d7ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.375044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074d44d-c192-492a-9164-1eeaa65d7ca4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.375587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 400/795] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:51.377486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adee37-f0f9-4df3-8155-2224ea77a8be None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.377967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adee37-f0f9-4df3-8155-2224ea77a8be None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.378418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adee37-f0f9-4df3-8155-2224ea77a8be None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.380239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 401/796] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:51.381126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adee37-f0f9-4df3-8155-2224ea77a8be None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:51.386136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074d44d-c192-492a-9164-1eeaa65d7ca4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:51.386136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074d44d-c192-492a-9164-1eeaa65d7ca4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3074d44d-c192-492a-9164-1eeaa65d7ca4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:51.401419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:51.402376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:51.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:51.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:51.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:51.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:51.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:51.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:51.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:51.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:51.403238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 396/797] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:51.404244 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c101250-2c5c-4fc9-b3da-595afa40c750 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.404469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c101250-2c5c-4fc9-b3da-595afa40c750 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.404743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c101250-2c5c-4fc9-b3da-595afa40c750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.430658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7c101250-2c5c-4fc9-b3da-595afa40c750 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:51.432664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7c101250-2c5c-4fc9-b3da-595afa40c750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:51.500792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:51.504315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adee37-f0f9-4df3-8155-2224ea77a8be None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:51.519313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 397/798] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:51.519313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c101250-2c5c-4fc9-b3da-595afa40c750 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:51.550907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:51.554772 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0b54f-b88f-45c3-95af-9383f9cfaa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:51.555763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0b54f-b88f-45c3-95af-9383f9cfaa4c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.556281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0b54f-b88f-45c3-95af-9383f9cfaa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.557078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0b54f-b88f-45c3-95af-9383f9cfaa4c None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.557980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0b54f-b88f-45c3-95af-9383f9cfaa4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:51.558462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0b54f-b88f-45c3-95af-9383f9cfaa4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:51.560073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0b54f-b88f-45c3-95af-9383f9cfaa4c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:51.572673 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 402/799] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:51.573891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0b54f-b88f-45c3-95af-9383f9cfaa4c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:51.639105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c101250-2c5c-4fc9-b3da-595afa40c750 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:51.677722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0b54f-b88f-45c3-95af-9383f9cfaa4c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:51.683574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817b69c-f68d-47b7-a878-c2994c2babed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:51.684300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817b69c-f68d-47b7-a878-c2994c2babed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.684584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817b69c-f68d-47b7-a878-c2994c2babed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.684862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817b69c-f68d-47b7-a878-c2994c2babed None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.685656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817b69c-f68d-47b7-a878-c2994c2babed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:51.685963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817b69c-f68d-47b7-a878-c2994c2babed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:51.687622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817b69c-f68d-47b7-a878-c2994c2babed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:51.705741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 398/800] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:51.706591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817b69c-f68d-47b7-a878-c2994c2babed None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:51.716485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8379c-1bf1-4fb9-ad08-8b27c06a0cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:51.717613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8379c-1bf1-4fb9-ad08-8b27c06a0cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.718117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8379c-1bf1-4fb9-ad08-8b27c06a0cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.718564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8379c-1bf1-4fb9-ad08-8b27c06a0cf8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.719418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8379c-1bf1-4fb9-ad08-8b27c06a0cf8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:51.720974 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8379c-1bf1-4fb9-ad08-8b27c06a0cf8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b1f8379c-1bf1-4fb9-ad08-8b27c06a0cf8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:51.733569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:51.734524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 02 03:21:51.735379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:51.738430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 403/801] 10.222.0.42 () {56 vars in 1129 bytes} [Sat Jul 2 03:21:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:51.739454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8379c-1bf1-4fb9-ad08-8b27c06a0cf8 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:51.771220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:51.803658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:51.820478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817b69c-f68d-47b7-a878-c2994c2babed None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:51.851363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8379c-1bf1-4fb9-ad08-8b27c06a0cf8 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:51.858672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423454d0-eadd-40ef-953a-5f518276cf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:51.859916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423454d0-eadd-40ef-953a-5f518276cf17 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.860208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423454d0-eadd-40ef-953a-5f518276cf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.860484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423454d0-eadd-40ef-953a-5f518276cf17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.868998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423454d0-eadd-40ef-953a-5f518276cf17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:51.870443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423454d0-eadd-40ef-953a-5f518276cf17 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-423454d0-eadd-40ef-953a-5f518276cf17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:51.884854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:51.885815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:51.886753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 399/802] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Jul 2 03:21:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:51.888040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423454d0-eadd-40ef-953a-5f518276cf17 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:51.892507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b6238-4eb4-42fb-8da7-781862f3cc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:51.893410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b6238-4eb4-42fb-8da7-781862f3cc38 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.893690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b6238-4eb4-42fb-8da7-781862f3cc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.893964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b6238-4eb4-42fb-8da7-781862f3cc38 None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/f37a95ba40774cfd89e6f397afa301a9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.895962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b6238-4eb4-42fb-8da7-781862f3cc38 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=f37a95ba40774cfd89e6f397afa301a9, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:51.897439 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b6238-4eb4-42fb-8da7-781862f3cc38 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:51.918689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 404/803] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:21:51 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/f37a95ba40774cfd89e6f397afa301a9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:51.918689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667534-caa1-4b30-8166-3466af9e9b83 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:51.918689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667534-caa1-4b30-8166-3466af9e9b83 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:51.918689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667534-caa1-4b30-8166-3466af9e9b83 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:51.918689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 405/804] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:51.918689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd667534-caa1-4b30-8166-3466af9e9b83 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:52.019727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423454d0-eadd-40ef-953a-5f518276cf17 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:52.031189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd667534-caa1-4b30-8166-3466af9e9b83 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:52.038790 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:52.053049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:52.073959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94e6fc-c7de-485d-9fce-bbb7b8f0f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:52.074640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94e6fc-c7de-485d-9fce-bbb7b8f0f57d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.074923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94e6fc-c7de-485d-9fce-bbb7b8f0f57d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.075197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94e6fc-c7de-485d-9fce-bbb7b8f0f57d None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.076026 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94e6fc-c7de-485d-9fce-bbb7b8f0f57d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:52.076372 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94e6fc-c7de-485d-9fce-bbb7b8f0f57d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:52.077564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e0274-1663-4245-a414-fb8e26b09830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:52.078031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94e6fc-c7de-485d-9fce-bbb7b8f0f57d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:52.078575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e0274-1663-4245-a414-fb8e26b09830 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.078996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e0274-1663-4245-a414-fb8e26b09830 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.079508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e0274-1663-4245-a414-fb8e26b09830 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.080707 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e0274-1663-4245-a414-fb8e26b09830 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:52.081186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e0274-1663-4245-a414-fb8e26b09830 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:52.082792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7e0274-1663-4245-a414-fb8e26b09830 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:52.087754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 400/805] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Jul 2 03:21:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:52.087754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268f8f-87fb-4607-80a1-3c41cc9e5478 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.087754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268f8f-87fb-4607-80a1-3c41cc9e5478 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.087754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268f8f-87fb-4607-80a1-3c41cc9e5478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.110736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-35268f8f-87fb-4607-80a1-3c41cc9e5478 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:52.113261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-35268f8f-87fb-4607-80a1-3c41cc9e5478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:52.113720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 406/806] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Jul 2 03:21:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:52.114791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e0274-1663-4245-a414-fb8e26b09830 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:52.207599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 401/807] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:52.215342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35268f8f-87fb-4607-80a1-3c41cc9e5478 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:52.231432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e0274-1663-4245-a414-fb8e26b09830 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:52.269821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75acd365-4821-4a59-bddb-fb4b7cf5b8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:52.270401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acd365-4821-4a59-bddb-fb4b7cf5b8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.270678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acd365-4821-4a59-bddb-fb4b7cf5b8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.270948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75acd365-4821-4a59-bddb-fb4b7cf5b8e1 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.271671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acd365-4821-4a59-bddb-fb4b7cf5b8e1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:52.271974 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acd365-4821-4a59-bddb-fb4b7cf5b8e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:52.273473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75acd365-4821-4a59-bddb-fb4b7cf5b8e1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:52.279282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 407/808] 10.222.0.42 () {56 vars in 1124 bytes} [Sat Jul 2 03:21:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:52.280400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fb2d3-1d3a-48a8-9adc-c5af7d45e508 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.280687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fb2d3-1d3a-48a8-9adc-c5af7d45e508 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.280958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fb2d3-1d3a-48a8-9adc-c5af7d45e508 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.286236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 408/809] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:52.287016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fb2d3-1d3a-48a8-9adc-c5af7d45e508 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:52.315037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35268f8f-87fb-4607-80a1-3c41cc9e5478 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:52.324396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:52.331322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:52.351675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55f8f2-8ea0-4f68-8c37-41fc21af803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:52.352394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55f8f2-8ea0-4f68-8c37-41fc21af803b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.352709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55f8f2-8ea0-4f68-8c37-41fc21af803b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.352991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55f8f2-8ea0-4f68-8c37-41fc21af803b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.396851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fb2d3-1d3a-48a8-9adc-c5af7d45e508 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:52.423971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55f8f2-8ea0-4f68-8c37-41fc21af803b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:52.425440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55f8f2-8ea0-4f68-8c37-41fc21af803b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:52.434211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a8423-d9c1-4207-b02e-9b9093dba7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:52.435003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a8423-d9c1-4207-b02e-9b9093dba7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.435286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a8423-d9c1-4207-b02e-9b9093dba7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.435593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a8423-d9c1-4207-b02e-9b9093dba7f7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.436245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a8423-d9c1-4207-b02e-9b9093dba7f7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:52.437625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a8423-d9c1-4207-b02e-9b9093dba7f7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-470a8423-d9c1-4207-b02e-9b9093dba7f7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:52.450850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:52.451923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:52.452752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 409/810] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:52 2022] GET /identity/v3/services/network => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:52.453620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a8423-d9c1-4207-b02e-9b9093dba7f7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:52.539446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 402/811] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:21:52.539446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c342b-8984-4cf3-8db1-30c79b5c6867 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.539446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c342b-8984-4cf3-8db1-30c79b5c6867 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.539446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c342b-8984-4cf3-8db1-30c79b5c6867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.558217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0b0c342b-8984-4cf3-8db1-30c79b5c6867 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:52.559446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0b0c342b-8984-4cf3-8db1-30c79b5c6867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:52.559566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a8423-d9c1-4207-b02e-9b9093dba7f7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:52.561117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:52.561621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:52.597911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364ecb9-7cfe-4999-b7f2-bc547b6e569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:52.598576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364ecb9-7cfe-4999-b7f2-bc547b6e569c None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.598858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364ecb9-7cfe-4999-b7f2-bc547b6e569c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.599135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364ecb9-7cfe-4999-b7f2-bc547b6e569c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.599901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364ecb9-7cfe-4999-b7f2-bc547b6e569c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:52.601266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364ecb9-7cfe-4999-b7f2-bc547b6e569c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:52.625112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 410/812] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Jul 2 03:21:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:21:52.625960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364ecb9-7cfe-4999-b7f2-bc547b6e569c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:52.655514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 403/813] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:52.656965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c342b-8984-4cf3-8db1-30c79b5c6867 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:52.751024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364ecb9-7cfe-4999-b7f2-bc547b6e569c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:52.792534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b13a59-cb92-4753-b1f2-f072ea076827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:52.793162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b13a59-cb92-4753-b1f2-f072ea076827 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.793449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b13a59-cb92-4753-b1f2-f072ea076827 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.795070 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b13a59-cb92-4753-b1f2-f072ea076827 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.795070 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b13a59-cb92-4753-b1f2-f072ea076827 None admin] RBAC: Adding query filter params (name=network) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:52.795070 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b13a59-cb92-4753-b1f2-f072ea076827 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:52.797016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b13a59-cb92-4753-b1f2-f072ea076827 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:52.801250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c342b-8984-4cf3-8db1-30c79b5c6867 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:52.802538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 411/814] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:21:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:52.803391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b13a59-cb92-4753-b1f2-f072ea076827 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:52.829209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:52.849469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb02a6-ad55-4cb9-a29e-04873cc4ea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:52.850598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb02a6-ad55-4cb9-a29e-04873cc4ea94 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.851099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb02a6-ad55-4cb9-a29e-04873cc4ea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.851616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb02a6-ad55-4cb9-a29e-04873cc4ea94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.854267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:52.867052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb02a6-ad55-4cb9-a29e-04873cc4ea94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:52.868747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb02a6-ad55-4cb9-a29e-04873cc4ea94 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-01fb02a6-ad55-4cb9-a29e-04873cc4ea94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:52.892107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:52.893094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:52.894430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:52.894430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:52.894430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:52.894430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:52.894430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 404/815] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:52.894430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb02a6-ad55-4cb9-a29e-04873cc4ea94 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:52.926932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b13a59-cb92-4753-b1f2-f072ea076827 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:52.969632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc1ac6-a9fe-4393-9a04-a3aa03af9e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:52.971021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc1ac6-a9fe-4393-9a04-a3aa03af9e7a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:52.971733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc1ac6-a9fe-4393-9a04-a3aa03af9e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:52.972582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc1ac6-a9fe-4393-9a04-a3aa03af9e7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:52.986451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc1ac6-a9fe-4393-9a04-a3aa03af9e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-8ccc1ac6-a9fe-4393-9a04-a3aa03af9e7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:52.997268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:52.998235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:52.999094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:53.000994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 412/816] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:53.006034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc1ac6-a9fe-4393-9a04-a3aa03af9e7a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:53.018040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb02a6-ad55-4cb9-a29e-04873cc4ea94 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:53.054425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c071d9-8c6e-4205-902a-4d36c40c4242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:53.055015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c071d9-8c6e-4205-902a-4d36c40c4242 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:53.055336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c071d9-8c6e-4205-902a-4d36c40c4242 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:53.055832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c071d9-8c6e-4205-902a-4d36c40c4242 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:53.057402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c071d9-8c6e-4205-902a-4d36c40c4242 None admin] RBAC: Adding query filter params (type=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:53.057726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c071d9-8c6e-4205-902a-4d36c40c4242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:53.059187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c071d9-8c6e-4205-902a-4d36c40c4242 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:53.066457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 405/817] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:21:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:53.067619 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c071d9-8c6e-4205-902a-4d36c40c4242 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:53.116672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:53.138806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:53.150574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc1ac6-a9fe-4393-9a04-a3aa03af9e7a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:53.192038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c071d9-8c6e-4205-902a-4d36c40c4242 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:53.196959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff02baa-d05e-4d73-ac9e-4903482498d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:53.197869 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff02baa-d05e-4d73-ac9e-4903482498d7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:53.198395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff02baa-d05e-4d73-ac9e-4903482498d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:53.198863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff02baa-d05e-4d73-ac9e-4903482498d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:53.199838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff02baa-d05e-4d73-ac9e-4903482498d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:53.200789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff02baa-d05e-4d73-ac9e-4903482498d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:53.202730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff02baa-d05e-4d73-ac9e-4903482498d7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:53.218847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 413/818] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:53.219984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff02baa-d05e-4d73-ac9e-4903482498d7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:53.232932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955b28b-374d-4c29-87c0-4a01aa473893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:53.233516 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955b28b-374d-4c29-87c0-4a01aa473893 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:53.233795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955b28b-374d-4c29-87c0-4a01aa473893 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:53.234068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955b28b-374d-4c29-87c0-4a01aa473893 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:53.234867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955b28b-374d-4c29-87c0-4a01aa473893 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:53.235174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955b28b-374d-4c29-87c0-4a01aa473893 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:53.236710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955b28b-374d-4c29-87c0-4a01aa473893 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:53.250519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 406/819] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:53.251439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955b28b-374d-4c29-87c0-4a01aa473893 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:53.344333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff02baa-d05e-4d73-ac9e-4903482498d7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:53.359793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955b28b-374d-4c29-87c0-4a01aa473893 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:53.373357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:53.390960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5ff30-8f87-47ba-bce7-0765d0e9a94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:53.392512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5ff30-8f87-47ba-bce7-0765d0e9a94d None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:53.393006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5ff30-8f87-47ba-bce7-0765d0e9a94d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:53.393550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5ff30-8f87-47ba-bce7-0765d0e9a94d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:53.394658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5ff30-8f87-47ba-bce7-0765d0e9a94d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:53.396314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7657a-18bb-482a-9357-15b676783c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:53.396698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5ff30-8f87-47ba-bce7-0765d0e9a94d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:53.397149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7657a-18bb-482a-9357-15b676783c08 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:53.397589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7657a-18bb-482a-9357-15b676783c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:53.397868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7657a-18bb-482a-9357-15b676783c08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:53.405720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7657a-18bb-482a-9357-15b676783c08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:53.407549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7657a-18bb-482a-9357-15b676783c08 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:53.413534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:53.423173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 414/820] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Jul 2 03:21:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-41c7657a-18bb-482a-9357-15b676783c08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:53.423536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:53.424421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:53.425345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:53.425345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:53.425345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:53.425345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:53.425345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:53.425345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:53.425345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:53.425345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:53.427451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5ff30-8f87-47ba-bce7-0765d0e9a94d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:53.429994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 407/821] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:53.431088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7657a-18bb-482a-9357-15b676783c08 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:53.557458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5ff30-8f87-47ba-bce7-0765d0e9a94d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:53.574015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7657a-18bb-482a-9357-15b676783c08 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:53.602708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe42233-170c-49df-93a1-d0068796a203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:53.603514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe42233-170c-49df-93a1-d0068796a203 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:53.603807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe42233-170c-49df-93a1-d0068796a203 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:53.604110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe42233-170c-49df-93a1-d0068796a203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:53.618939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b37d1-51f1-44f1-a3fd-7496f44b20d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:53.620014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b37d1-51f1-44f1-a3fd-7496f44b20d7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:53.621173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b37d1-51f1-44f1-a3fd-7496f44b20d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:53.621700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4b37d1-51f1-44f1-a3fd-7496f44b20d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:53.622707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b37d1-51f1-44f1-a3fd-7496f44b20d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:53.623219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b37d1-51f1-44f1-a3fd-7496f44b20d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:53.625499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4b37d1-51f1-44f1-a3fd-7496f44b20d7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:53.630642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe42233-170c-49df-93a1-d0068796a203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:53.632158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe42233-170c-49df-93a1-d0068796a203 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:53.640696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 408/822] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:53.651423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b37d1-51f1-44f1-a3fd-7496f44b20d7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6fe42233-170c-49df-93a1-d0068796a203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:53.663817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:53.664759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:53.665669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:53.665669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:53.665669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:53.665669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:53.665669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 415/823] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:53.671076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe42233-170c-49df-93a1-d0068796a203 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:53.688629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:53.724242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:53.768055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4b37d1-51f1-44f1-a3fd-7496f44b20d7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:53.781304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe42233-170c-49df-93a1-d0068796a203 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:53.809192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04185ff4-ec00-4217-92c1-e166d9ac1eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:53.810062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04185ff4-ec00-4217-92c1-e166d9ac1eaa None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:53.810344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04185ff4-ec00-4217-92c1-e166d9ac1eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:53.810711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04185ff4-ec00-4217-92c1-e166d9ac1eaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:53.813067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04185ff4-ec00-4217-92c1-e166d9ac1eaa None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=8b7bc20034974816905978dd365993f8) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:53.813067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04185ff4-ec00-4217-92c1-e166d9ac1eaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:53.813345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04185ff4-ec00-4217-92c1-e166d9ac1eaa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:53.820125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eb349-5230-4320-a535-cebb3fa5cad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:53.820401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 409/824] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:53 2022] GET /identity/v3/role_assignments?user.id=8b7bc20034974816905978dd365993f8&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:53.820789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eb349-5230-4320-a535-cebb3fa5cad3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:53.821125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eb349-5230-4320-a535-cebb3fa5cad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:53.821447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eb349-5230-4320-a535-cebb3fa5cad3 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:53.822176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3eb349-5230-4320-a535-cebb3fa5cad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:53.822481 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3eb349-5230-4320-a535-cebb3fa5cad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:53.826761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3eb349-5230-4320-a535-cebb3fa5cad3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:53.842258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 416/825] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:53.848501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04185ff4-ec00-4217-92c1-e166d9ac1eaa None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:53.960865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:53.967542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04185ff4-ec00-4217-92c1-e166d9ac1eaa None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:54.004725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a033c8a-6bdb-470a-9495-a0fef4089fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:54.005964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a033c8a-6bdb-470a-9495-a0fef4089fbc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.006446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a033c8a-6bdb-470a-9495-a0fef4089fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.006918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a033c8a-6bdb-470a-9495-a0fef4089fbc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.015205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a033c8a-6bdb-470a-9495-a0fef4089fbc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:54.016983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a033c8a-6bdb-470a-9495-a0fef4089fbc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9a033c8a-6bdb-470a-9495-a0fef4089fbc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:54.032075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:54.033089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:21:54.034237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:54.036240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 410/826] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:21:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:54.042499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eb349-5230-4320-a535-cebb3fa5cad3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:54.147548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eb349-5230-4320-a535-cebb3fa5cad3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:54.155473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:54.183268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00ca9a-d652-4443-8f07-f6641985d7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:54.183974 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00ca9a-d652-4443-8f07-f6641985d7fc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.184257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00ca9a-d652-4443-8f07-f6641985d7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.184536 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00ca9a-d652-4443-8f07-f6641985d7fc None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.185379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00ca9a-d652-4443-8f07-f6641985d7fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:54.185689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00ca9a-d652-4443-8f07-f6641985d7fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:54.187153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00ca9a-d652-4443-8f07-f6641985d7fc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:54.205864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 417/827] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Jul 2 03:21:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:54.212519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a033c8a-6bdb-470a-9495-a0fef4089fbc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:54.248181 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dab0d-efef-4c86-bb94-3f8c090f7055 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.248502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dab0d-efef-4c86-bb94-3f8c090f7055 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.248737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dab0d-efef-4c86-bb94-3f8c090f7055 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.250366 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 418/828] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:54.255402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d09f60-d0d1-4edb-bbd0-433f6f464c1a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.255693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d09f60-d0d1-4edb-bbd0-433f6f464c1a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.255973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d09f60-d0d1-4edb-bbd0-433f6f464c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.272979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-88d09f60-d0d1-4edb-bbd0-433f6f464c1a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:54.274321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-88d09f60-d0d1-4edb-bbd0-433f6f464c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:54.318604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a033c8a-6bdb-470a-9495-a0fef4089fbc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:54.347562 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 419/829] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:21:54.353995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedffccd-1e97-42ca-8f4c-c4a4c2319355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:54.354800 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedffccd-1e97-42ca-8f4c-c4a4c2319355 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.355080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedffccd-1e97-42ca-8f4c-c4a4c2319355 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.355378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedffccd-1e97-42ca-8f4c-c4a4c2319355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.366602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d09f60-d0d1-4edb-bbd0-433f6f464c1a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:54.366880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedffccd-1e97-42ca-8f4c-c4a4c2319355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:54.368398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedffccd-1e97-42ca-8f4c-c4a4c2319355 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-dedffccd-1e97-42ca-8f4c-c4a4c2319355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:54.386507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:54.387461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:54.388353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:54.388353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:54.388353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:54.388353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:54.388353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 411/830] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:54.393542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedffccd-1e97-42ca-8f4c-c4a4c2319355 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:54.409910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:54.461745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:54.468956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d09f60-d0d1-4edb-bbd0-433f6f464c1a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:54.501092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedffccd-1e97-42ca-8f4c-c4a4c2319355 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:54.503343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f73a2-098c-44cd-a3b3-1a46c44e127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:54.504821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f73a2-098c-44cd-a3b3-1a46c44e127f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.504821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f73a2-098c-44cd-a3b3-1a46c44e127f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.504821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f73a2-098c-44cd-a3b3-1a46c44e127f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.515857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f73a2-098c-44cd-a3b3-1a46c44e127f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c99f73a2-098c-44cd-a3b3-1a46c44e127f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:54.523466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:54.524924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:21:54.525946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:54.531808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 420/831] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:54.532739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f73a2-098c-44cd-a3b3-1a46c44e127f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:54.541090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc12e4-453e-4f08-8bbd-a2b088fcbc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:54.541960 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc12e4-453e-4f08-8bbd-a2b088fcbc97 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.542415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc12e4-453e-4f08-8bbd-a2b088fcbc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.542921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc12e4-453e-4f08-8bbd-a2b088fcbc97 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.543923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc12e4-453e-4f08-8bbd-a2b088fcbc97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:54.544398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc12e4-453e-4f08-8bbd-a2b088fcbc97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:54.546041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc12e4-453e-4f08-8bbd-a2b088fcbc97 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:54.564666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 412/832] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:54.565986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc12e4-453e-4f08-8bbd-a2b088fcbc97 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:54.634262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f73a2-098c-44cd-a3b3-1a46c44e127f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:54.635635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:54.669385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d6e2c-7d8b-47f7-9216-4ff982a3e670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:54.670343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d6e2c-7d8b-47f7-9216-4ff982a3e670 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.670773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d6e2c-7d8b-47f7-9216-4ff982a3e670 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.671195 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2d6e2c-7d8b-47f7-9216-4ff982a3e670 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.672118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d6e2c-7d8b-47f7-9216-4ff982a3e670 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:54.672343 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc12e4-453e-4f08-8bbd-a2b088fcbc97 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:54.672722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d6e2c-7d8b-47f7-9216-4ff982a3e670 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:54.674255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2d6e2c-7d8b-47f7-9216-4ff982a3e670 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:54.688001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 421/833] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:21:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:54.689170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d6e2c-7d8b-47f7-9216-4ff982a3e670 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:54.707873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:54.711128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6212026-0b90-4735-83a7-520e34c6b821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:54.712187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6212026-0b90-4735-83a7-520e34c6b821 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.712533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6212026-0b90-4735-83a7-520e34c6b821 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.712769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6212026-0b90-4735-83a7-520e34c6b821 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.721083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6212026-0b90-4735-83a7-520e34c6b821 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:54.722641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6212026-0b90-4735-83a7-520e34c6b821 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d6212026-0b90-4735-83a7-520e34c6b821 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:54.737339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:54.738247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:54.739110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:54.739110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:54.739110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:54.739110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:54.739110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:54.739110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:54.739110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:54.739110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:54.743672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 413/834] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:54.744622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6212026-0b90-4735-83a7-520e34c6b821 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:54.794727 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2d6e2c-7d8b-47f7-9216-4ff982a3e670 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:54.830458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3a2be-df57-4e5d-b3c0-4305e372bdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:54.831258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3a2be-df57-4e5d-b3c0-4305e372bdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.831569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3a2be-df57-4e5d-b3c0-4305e372bdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.831845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3a2be-df57-4e5d-b3c0-4305e372bdb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.843240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3a2be-df57-4e5d-b3c0-4305e372bdb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:54.844818 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3a2be-df57-4e5d-b3c0-4305e372bdb9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:54.850225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6212026-0b90-4735-83a7-520e34c6b821 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c6e3a2be-df57-4e5d-b3c0-4305e372bdb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:54.864373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:54.865345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:54.866247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:54.866247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:54.866247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:54.866247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:54.866247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 422/835] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:54.867196 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4df06-fcce-4cef-92fe-7d7aacf53b6e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.867548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4df06-fcce-4cef-92fe-7d7aacf53b6e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.867646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4df06-fcce-4cef-92fe-7d7aacf53b6e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.869121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 423/836] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:54.872165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4df06-fcce-4cef-92fe-7d7aacf53b6e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:54.885906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:54.895639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356007e3-0f31-4a0d-9a20-0cb9ec75de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:54.896734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356007e3-0f31-4a0d-9a20-0cb9ec75de6a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:54.897017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356007e3-0f31-4a0d-9a20-0cb9ec75de6a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:54.897553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356007e3-0f31-4a0d-9a20-0cb9ec75de6a None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:54.898361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356007e3-0f31-4a0d-9a20-0cb9ec75de6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:54.898675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356007e3-0f31-4a0d-9a20-0cb9ec75de6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:54.900839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356007e3-0f31-4a0d-9a20-0cb9ec75de6a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:54.947321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:54.988276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4df06-fcce-4cef-92fe-7d7aacf53b6e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:55.026132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d74c7-49c7-4c50-844a-1c1fb611de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:55.026798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d74c7-49c7-4c50-844a-1c1fb611de05 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:55.027074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d74c7-49c7-4c50-844a-1c1fb611de05 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:55.027449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d74c7-49c7-4c50-844a-1c1fb611de05 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:55.028176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d74c7-49c7-4c50-844a-1c1fb611de05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:55.028477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d74c7-49c7-4c50-844a-1c1fb611de05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:55.029851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d74c7-49c7-4c50-844a-1c1fb611de05 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:55.035375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 414/837] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:55.037227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648efd3-85b3-49ae-8106-eb95166baf73 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:55.037528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648efd3-85b3-49ae-8106-eb95166baf73 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:55.037808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648efd3-85b3-49ae-8106-eb95166baf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:55.042462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 424/838] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:55.043318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d74c7-49c7-4c50-844a-1c1fb611de05 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:55.062832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8648efd3-85b3-49ae-8106-eb95166baf73 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:55.064078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8648efd3-85b3-49ae-8106-eb95166baf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:55.132525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 415/839] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:55.133801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648efd3-85b3-49ae-8106-eb95166baf73 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:55.169397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d74c7-49c7-4c50-844a-1c1fb611de05 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:55.196278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:55.205600 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d16eb2-64cd-4c25-ba85-3f57a55a1586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:55.206502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d16eb2-64cd-4c25-ba85-3f57a55a1586 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:55.206781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d16eb2-64cd-4c25-ba85-3f57a55a1586 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:55.207051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d16eb2-64cd-4c25-ba85-3f57a55a1586 None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/2fcefa2d73a84415875357ef03e2c8da/roles/4a54555205fa406897a3abc7871ee1aa` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:55.222280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d16eb2-64cd-4c25-ba85-3f57a55a1586 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=2fcefa2d73a84415875357ef03e2c8da, role_id=4a54555205fa406897a3abc7871ee1aa)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:55.223773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d16eb2-64cd-4c25-ba85-3f57a55a1586 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:55.247413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 425/840] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:21:55 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/2fcefa2d73a84415875357ef03e2c8da/roles/4a54555205fa406897a3abc7871ee1aa => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:21:55.247413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d16eb2-64cd-4c25-ba85-3f57a55a1586 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:55.270577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648efd3-85b3-49ae-8106-eb95166baf73 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:55.292776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:55.313561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55949219-ab52-4798-872b-dfc8d88cdb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:55.314526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55949219-ab52-4798-872b-dfc8d88cdb41 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:55.314899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55949219-ab52-4798-872b-dfc8d88cdb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:55.315308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55949219-ab52-4798-872b-dfc8d88cdb41 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:55.324274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55949219-ab52-4798-872b-dfc8d88cdb41 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:55.326573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55949219-ab52-4798-872b-dfc8d88cdb41 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-55949219-ab52-4798-872b-dfc8d88cdb41 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:55.345373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:55.346362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 02 03:21:55.347334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:55.351618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 416/841] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:21:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:55.352651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55949219-ab52-4798-872b-dfc8d88cdb41 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:55.389944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d16eb2-64cd-4c25-ba85-3f57a55a1586 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:55.435701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e556c-0490-4b82-991d-1f452798ec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:55.436759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e556c-0490-4b82-991d-1f452798ec60 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:55.437054 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e556c-0490-4b82-991d-1f452798ec60 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:55.437328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e556c-0490-4b82-991d-1f452798ec60 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:55.437982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e556c-0490-4b82-991d-1f452798ec60 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:55.439415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e556c-0490-4b82-991d-1f452798ec60 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e87e556c-0490-4b82-991d-1f452798ec60 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:55.453351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:55.454753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:55.455751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 426/842] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:55.458887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e556c-0490-4b82-991d-1f452798ec60 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:55.482025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55949219-ab52-4798-872b-dfc8d88cdb41 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:55.502604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:55.529599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069a51-5551-4ccf-a6de-45570a5ec385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:55.530529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069a51-5551-4ccf-a6de-45570a5ec385 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:55.530948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069a51-5551-4ccf-a6de-45570a5ec385 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:55.531389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4069a51-5551-4ccf-a6de-45570a5ec385 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:55.532607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069a51-5551-4ccf-a6de-45570a5ec385 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:55.533644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069a51-5551-4ccf-a6de-45570a5ec385 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:55.535521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4069a51-5551-4ccf-a6de-45570a5ec385 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:55.564869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 417/843] 10.222.0.42 () {56 vars in 1135 bytes} [Sat Jul 2 03:21:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:55.566293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069a51-5551-4ccf-a6de-45570a5ec385 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:55.575431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e556c-0490-4b82-991d-1f452798ec60 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:55.612363 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:55.617041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450334c6-f8a1-4f5a-94dc-62beeae921ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:55.618182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450334c6-f8a1-4f5a-94dc-62beeae921ff None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:55.618676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450334c6-f8a1-4f5a-94dc-62beeae921ff None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:55.619159 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450334c6-f8a1-4f5a-94dc-62beeae921ff None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:55.620597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450334c6-f8a1-4f5a-94dc-62beeae921ff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:55.621275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450334c6-f8a1-4f5a-94dc-62beeae921ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:55.623568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450334c6-f8a1-4f5a-94dc-62beeae921ff None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:55.630816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 427/844] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:55.636923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450334c6-f8a1-4f5a-94dc-62beeae921ff None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:55.696311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4069a51-5551-4ccf-a6de-45570a5ec385 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:55.745586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640c1dd-433f-47b3-b398-b60f21b96bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:55.747111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640c1dd-433f-47b3-b398-b60f21b96bbc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:55.747539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640c1dd-433f-47b3-b398-b60f21b96bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:55.747817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640c1dd-433f-47b3-b398-b60f21b96bbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:55.761909 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450334c6-f8a1-4f5a-94dc-62beeae921ff None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:55.769925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:55.800166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccec3d-53c1-4bf5-8b00-74bd00af7907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:55.800998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccec3d-53c1-4bf5-8b00-74bd00af7907 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:55.801383 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccec3d-53c1-4bf5-8b00-74bd00af7907 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:55.801765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccec3d-53c1-4bf5-8b00-74bd00af7907 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:55.802583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccec3d-53c1-4bf5-8b00-74bd00af7907 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:55.803168 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccec3d-53c1-4bf5-8b00-74bd00af7907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:55.804789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccec3d-53c1-4bf5-8b00-74bd00af7907 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:55.815359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 428/845] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:21:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:55.839626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640c1dd-433f-47b3-b398-b60f21b96bbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:55.841135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640c1dd-433f-47b3-b398-b60f21b96bbc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:55.881239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:55.952987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 418/846] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:21:55.959235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccec3d-53c1-4bf5-8b00-74bd00af7907 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:56.016259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf86f79-4f49-42ca-a8a7-4f3a447d0090 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.016564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf86f79-4f49-42ca-a8a7-4f3a447d0090 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.016778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf86f79-4f49-42ca-a8a7-4f3a447d0090 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.019421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 419/847] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:56.023579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13639ab2-2780-4b01-a09b-6734b865a38a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.023865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13639ab2-2780-4b01-a09b-6734b865a38a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.024173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13639ab2-2780-4b01-a09b-6734b865a38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.043578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-13639ab2-2780-4b01-a09b-6734b865a38a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:56.044783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-13639ab2-2780-4b01-a09b-6734b865a38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:56.072832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccec3d-53c1-4bf5-8b00-74bd00af7907 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:56.116551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21dc88-62a5-4d0a-a241-c67ca04c31a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:56.118098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21dc88-62a5-4d0a-a241-c67ca04c31a5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.118824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21dc88-62a5-4d0a-a241-c67ca04c31a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.119324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21dc88-62a5-4d0a-a241-c67ca04c31a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.135470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 420/848] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:21:56.138526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21dc88-62a5-4d0a-a241-c67ca04c31a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:56.140277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21dc88-62a5-4d0a-a241-c67ca04c31a5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:56.152254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13639ab2-2780-4b01-a09b-6734b865a38a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0c21dc88-62a5-4d0a-a241-c67ca04c31a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:56.159573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:56.160503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:56.161382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:56.161382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:56.161382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:56.161382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:56.163851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 429/849] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:56.169207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21dc88-62a5-4d0a-a241-c67ca04c31a5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:56.200582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:56.233273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:56.271675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13639ab2-2780-4b01-a09b-6734b865a38a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:56.284656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21dc88-62a5-4d0a-a241-c67ca04c31a5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:56.309331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef68ec-fcda-47ec-b28a-980b909244f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:56.310595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef68ec-fcda-47ec-b28a-980b909244f8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.310882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef68ec-fcda-47ec-b28a-980b909244f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.311158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef68ec-fcda-47ec-b28a-980b909244f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.311866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef68ec-fcda-47ec-b28a-980b909244f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:56.313252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef68ec-fcda-47ec-b28a-980b909244f8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:56.322039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f19e49-b9ce-48d8-90c3-c6b2343b1e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:56.322741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f19e49-b9ce-48d8-90c3-c6b2343b1e2d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.323024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f19e49-b9ce-48d8-90c3-c6b2343b1e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.323313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f19e49-b9ce-48d8-90c3-c6b2343b1e2d None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.324051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f19e49-b9ce-48d8-90c3-c6b2343b1e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:56.324392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f19e49-b9ce-48d8-90c3-c6b2343b1e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:56.325765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f19e49-b9ce-48d8-90c3-c6b2343b1e2d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a7ef68ec-fcda-47ec-b28a-980b909244f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:56.326542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:56.327459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 02 03:21:56.328284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:56.331668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 421/850] 10.222.0.42 () {56 vars in 1132 bytes} [Sat Jul 2 03:21:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:56.333873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef68ec-fcda-47ec-b28a-980b909244f8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:56.344009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 430/851] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:56.344924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f19e49-b9ce-48d8-90c3-c6b2343b1e2d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:56.438397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef68ec-fcda-47ec-b28a-980b909244f8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:56.439093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:56.464475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f19e49-b9ce-48d8-90c3-c6b2343b1e2d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:56.472907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:56.479465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd427981-8875-4a1a-8712-2a89cb9ea991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:56.480150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd427981-8875-4a1a-8712-2a89cb9ea991 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.480436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd427981-8875-4a1a-8712-2a89cb9ea991 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.480716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd427981-8875-4a1a-8712-2a89cb9ea991 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.481433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd427981-8875-4a1a-8712-2a89cb9ea991 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:56.481743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd427981-8875-4a1a-8712-2a89cb9ea991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:56.483127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd427981-8875-4a1a-8712-2a89cb9ea991 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:56.492701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 422/852] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:21:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:56.494209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd427981-8875-4a1a-8712-2a89cb9ea991 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:56.506327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0405632-0bc4-4513-975e-23559cec9b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:56.507762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0405632-0bc4-4513-975e-23559cec9b1f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.508256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0405632-0bc4-4513-975e-23559cec9b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.508774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0405632-0bc4-4513-975e-23559cec9b1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.516550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0405632-0bc4-4513-975e-23559cec9b1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:56.518279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0405632-0bc4-4513-975e-23559cec9b1f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a0405632-0bc4-4513-975e-23559cec9b1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:56.533069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:56.533945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:56.534828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:56.534828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:56.534828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:56.534828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:56.534828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:56.534828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:56.534828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:56.534828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:56.537444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 431/853] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:56.544556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0405632-0bc4-4513-975e-23559cec9b1f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:56.599917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd427981-8875-4a1a-8712-2a89cb9ea991 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:56.637598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f55ac-d87a-456e-9462-c9528a14d939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:56.638624 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f55ac-d87a-456e-9462-c9528a14d939 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.639110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f55ac-d87a-456e-9462-c9528a14d939 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.639636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f55ac-d87a-456e-9462-c9528a14d939 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.640648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f55ac-d87a-456e-9462-c9528a14d939 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:56.641155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f55ac-d87a-456e-9462-c9528a14d939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:56.642743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f55ac-d87a-456e-9462-c9528a14d939 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:56.655434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 423/854] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:21:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:56.667161 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0405632-0bc4-4513-975e-23559cec9b1f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:56.705806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8efe6-6e90-4f40-8ac6-acd281b22a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:56.706745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8efe6-6e90-4f40-8ac6-acd281b22a56 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.707168 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8efe6-6e90-4f40-8ac6-acd281b22a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.707668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8efe6-6e90-4f40-8ac6-acd281b22a56 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.708629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8efe6-6e90-4f40-8ac6-acd281b22a56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:56.709079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8efe6-6e90-4f40-8ac6-acd281b22a56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:56.710702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8efe6-6e90-4f40-8ac6-acd281b22a56 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:56.724988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 432/855] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:56.731654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f55ac-d87a-456e-9462-c9528a14d939 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:56.796359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:56.842067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f55ac-d87a-456e-9462-c9528a14d939 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:56.877419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f3bbe-8c0d-4f82-a0ca-e26415f1898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:56.878324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3bbe-8c0d-4f82-a0ca-e26415f1898c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:56.878819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3bbe-8c0d-4f82-a0ca-e26415f1898c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:56.879245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3bbe-8c0d-4f82-a0ca-e26415f1898c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:56.880288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f3bbe-8c0d-4f82-a0ca-e26415f1898c None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=f37a95ba40774cfd89e6f397afa301a9) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:56.880735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f3bbe-8c0d-4f82-a0ca-e26415f1898c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:56.882311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f3bbe-8c0d-4f82-a0ca-e26415f1898c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:56.889313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 424/856] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:56 2022] GET /identity/v3/role_assignments?user.id=f37a95ba40774cfd89e6f397afa301a9&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:57.324413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32915a90-1e57-4434-9ec3-4d60234de344 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:57.324936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32915a90-1e57-4434-9ec3-4d60234de344 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:57.324936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32915a90-1e57-4434-9ec3-4d60234de344 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:57.326558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 433/857] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:57.332062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbcc24-19b1-4b24-801c-58bb753e684b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:57.332496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbcc24-19b1-4b24-801c-58bb753e684b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:57.332693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbcc24-19b1-4b24-801c-58bb753e684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:57.350542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-29bbcc24-19b1-4b24-801c-58bb753e684b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:57.351768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29bbcc24-19b1-4b24-801c-58bb753e684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:57.427824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 425/858] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:21:57.451026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32915a90-1e57-4434-9ec3-4d60234de344 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:57.454025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:57.554065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32915a90-1e57-4434-9ec3-4d60234de344 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:57.589321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ba2e5-c98e-47a9-8b85-7577645de7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:57.590123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ba2e5-c98e-47a9-8b85-7577645de7d5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:57.590404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ba2e5-c98e-47a9-8b85-7577645de7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:57.590681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ba2e5-c98e-47a9-8b85-7577645de7d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:57.601355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ba2e5-c98e-47a9-8b85-7577645de7d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f83ba2e5-c98e-47a9-8b85-7577645de7d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:21:57.609199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:57.610118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:57.611390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 434/859] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:21:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:57.616582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbcc24-19b1-4b24-801c-58bb753e684b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:57.671171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:57.719699 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbcc24-19b1-4b24-801c-58bb753e684b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:57.754577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aa30c-e182-4a23-9f2a-76b981d0b62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:57.755249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aa30c-e182-4a23-9f2a-76b981d0b62c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:57.755557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aa30c-e182-4a23-9f2a-76b981d0b62c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:57.755832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aa30c-e182-4a23-9f2a-76b981d0b62c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:57.756577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25aa30c-e182-4a23-9f2a-76b981d0b62c None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:57.756881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25aa30c-e182-4a23-9f2a-76b981d0b62c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:57.758255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25aa30c-e182-4a23-9f2a-76b981d0b62c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:57.767035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 426/860] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:21:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:57.772546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ba2e5-c98e-47a9-8b85-7577645de7d5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:57.836323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:57.887688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ba2e5-c98e-47a9-8b85-7577645de7d5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:57.914231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e37fc-041c-40b1-acbb-e8b855279a89 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:57.914605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e37fc-041c-40b1-acbb-e8b855279a89 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:57.915057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e37fc-041c-40b1-acbb-e8b855279a89 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:57.916728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 427/861] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:57.921431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a7400-0d28-4b03-8e8a-50734fdc6e48 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:57.921715 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a7400-0d28-4b03-8e8a-50734fdc6e48 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:57.921991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a7400-0d28-4b03-8e8a-50734fdc6e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:57.930416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb87e9-4493-4f61-b5e5-c69dda1d8db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:57.931893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb87e9-4493-4f61-b5e5-c69dda1d8db5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:57.932374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb87e9-4493-4f61-b5e5-c69dda1d8db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:57.932841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb87e9-4493-4f61-b5e5-c69dda1d8db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:57.939222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d41a7400-0d28-4b03-8e8a-50734fdc6e48 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:57.940451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d41a7400-0d28-4b03-8e8a-50734fdc6e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:57.944562 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb87e9-4493-4f61-b5e5-c69dda1d8db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:57.946310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb87e9-4493-4f61-b5e5-c69dda1d8db5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1afb87e9-4493-4f61-b5e5-c69dda1d8db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:57.963777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:57.964705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:57.965611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:57.965611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:57.965611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:57.965611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:57.967818 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 435/862] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:57.973375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb87e9-4493-4f61-b5e5-c69dda1d8db5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:58.023314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 428/863] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:21:58.038560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a7400-0d28-4b03-8e8a-50734fdc6e48 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:58.081605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb87e9-4493-4f61-b5e5-c69dda1d8db5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:58.086116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:58.100174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:58.123500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3af34e-1be8-4f5b-8894-34f5493dd871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:58.124218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3af34e-1be8-4f5b-8894-34f5493dd871 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:58.124508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3af34e-1be8-4f5b-8894-34f5493dd871 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:58.124787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3af34e-1be8-4f5b-8894-34f5493dd871 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:58.125515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3af34e-1be8-4f5b-8894-34f5493dd871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:58.125824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3af34e-1be8-4f5b-8894-34f5493dd871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:58.127524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3af34e-1be8-4f5b-8894-34f5493dd871 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:58.146117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a7400-0d28-4b03-8e8a-50734fdc6e48 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:58.184974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324be5cb-4c52-441c-a497-a7c2e3fb858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:58.186095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324be5cb-4c52-441c-a497-a7c2e3fb858a None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:58.186579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324be5cb-4c52-441c-a497-a7c2e3fb858a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:58.187074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324be5cb-4c52-441c-a497-a7c2e3fb858a None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:58.188040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324be5cb-4c52-441c-a497-a7c2e3fb858a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:58.189670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324be5cb-4c52-441c-a497-a7c2e3fb858a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:58.207015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 429/864] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:21:58 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:21:58.267453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 436/865] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:58.273072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324be5cb-4c52-441c-a497-a7c2e3fb858a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:58.395117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:58.401919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324be5cb-4c52-441c-a497-a7c2e3fb858a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:58.438686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54caea1-7a42-474e-9208-b26792e9c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:58.439556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54caea1-7a42-474e-9208-b26792e9c79c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:58.439837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54caea1-7a42-474e-9208-b26792e9c79c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:58.440107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54caea1-7a42-474e-9208-b26792e9c79c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:58.448196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54caea1-7a42-474e-9208-b26792e9c79c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:58.449650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54caea1-7a42-474e-9208-b26792e9c79c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a54caea1-7a42-474e-9208-b26792e9c79c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:21:58.464571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:58.465533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:58.466452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 430/866] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:21:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:58.473689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3af34e-1be8-4f5b-8894-34f5493dd871 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:58.576007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3af34e-1be8-4f5b-8894-34f5493dd871 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:58.611469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f9f23-9d73-4eb5-bd2e-52183eb3814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:58.612576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f9f23-9d73-4eb5-bd2e-52183eb3814b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:58.613056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f9f23-9d73-4eb5-bd2e-52183eb3814b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:58.613522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f9f23-9d73-4eb5-bd2e-52183eb3814b None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:58.614543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f9f23-9d73-4eb5-bd2e-52183eb3814b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:58.615045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f9f23-9d73-4eb5-bd2e-52183eb3814b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:58.616704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f9f23-9d73-4eb5-bd2e-52183eb3814b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:58.635497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 437/867] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Jul 2 03:21:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:58.641403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54caea1-7a42-474e-9208-b26792e9c79c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:58.751764 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54caea1-7a42-474e-9208-b26792e9c79c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:58.788366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22814e75-01ae-4e0e-b3f9-5245cba031b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:58.789020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22814e75-01ae-4e0e-b3f9-5245cba031b9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:58.789302 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22814e75-01ae-4e0e-b3f9-5245cba031b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:58.789651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22814e75-01ae-4e0e-b3f9-5245cba031b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:58.818362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:58.844379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00708a19-0989-46b6-a2c1-72686a8fa3c5 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:58.844557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00708a19-0989-46b6-a2c1-72686a8fa3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:58.844950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00708a19-0989-46b6-a2c1-72686a8fa3c5 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:58.846533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 438/868] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:21:58.852114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f5588-4b59-4f3a-861f-9a7464090e07 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:58.852511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f5588-4b59-4f3a-861f-9a7464090e07 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:58.852811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f5588-4b59-4f3a-861f-9a7464090e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:58.865132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22814e75-01ae-4e0e-b3f9-5245cba031b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:58.866795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22814e75-01ae-4e0e-b3f9-5245cba031b9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:58.871800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-189f5588-4b59-4f3a-861f-9a7464090e07 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:58.873110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-189f5588-4b59-4f3a-861f-9a7464090e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:58.905150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:58.962447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 439/869] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:58 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:21:58.974036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 431/870] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:21:58 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:21:58.980456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f5588-4b59-4f3a-861f-9a7464090e07 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:58.984360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22814e75-01ae-4e0e-b3f9-5245cba031b9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:59.047736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:59.096134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22814e75-01ae-4e0e-b3f9-5245cba031b9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:59.115964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f5588-4b59-4f3a-861f-9a7464090e07 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:59.134533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3ed23-4642-47bf-a877-1e3a0245c912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:59.135219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3ed23-4642-47bf-a877-1e3a0245c912 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.135617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3ed23-4642-47bf-a877-1e3a0245c912 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.135807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3ed23-4642-47bf-a877-1e3a0245c912 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.136533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3ed23-4642-47bf-a877-1e3a0245c912 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:59.137911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3ed23-4642-47bf-a877-1e3a0245c912 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:59.151768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 432/871] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:21:58 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:21:59.153347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05002b4-1163-4bd3-9814-4e6badd2c230 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.153827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05002b4-1163-4bd3-9814-4e6badd2c230 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.154317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05002b4-1163-4bd3-9814-4e6badd2c230 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.156092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 433/872] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:21:59.163335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690bc1-5b94-4cd7-8c4a-a65a6e12835b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.163335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690bc1-5b94-4cd7-8c4a-a65a6e12835b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.163335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28690bc1-5b94-4cd7-8c4a-a65a6e12835b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.173544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e30df0-ad74-4ffb-8a7f-3fd749e97da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:59.174600 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e30df0-ad74-4ffb-8a7f-3fd749e97da0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.174959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e30df0-ad74-4ffb-8a7f-3fd749e97da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.175323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e30df0-ad74-4ffb-8a7f-3fd749e97da0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.187030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-28690bc1-5b94-4cd7-8c4a-a65a6e12835b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:21:59.188687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-28690bc1-5b94-4cd7-8c4a-a65a6e12835b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:21:59.189782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e30df0-ad74-4ffb-8a7f-3fd749e97da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:59.191631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e30df0-ad74-4ffb-8a7f-3fd749e97da0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-84e30df0-ad74-4ffb-8a7f-3fd749e97da0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:21:59.221306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:59.222330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:59.223222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:21:59.223222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:21:59.223222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:21:59.223222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:59.223222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 440/873] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:59.228475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e30df0-ad74-4ffb-8a7f-3fd749e97da0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:59.248354 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:59.281387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 434/874] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:21:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:21:59.305651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28690bc1-5b94-4cd7-8c4a-a65a6e12835b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:59.348884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e30df0-ad74-4ffb-8a7f-3fd749e97da0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:59.365761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:59.388655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89177c-4945-4a2b-843c-6fdc6a47d7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:59.389425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89177c-4945-4a2b-843c-6fdc6a47d7be None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.389708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89177c-4945-4a2b-843c-6fdc6a47d7be None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.390084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89177c-4945-4a2b-843c-6fdc6a47d7be None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.390803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89177c-4945-4a2b-843c-6fdc6a47d7be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:59.391114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89177c-4945-4a2b-843c-6fdc6a47d7be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:59.392514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89177c-4945-4a2b-843c-6fdc6a47d7be None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:59.406356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 441/875] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:59.412899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89177c-4945-4a2b-843c-6fdc6a47d7be None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:59.428560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28690bc1-5b94-4cd7-8c4a-a65a6e12835b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:59.469849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fbaa1-0d1a-4815-8517-58b27b6f728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:59.470498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fbaa1-0d1a-4815-8517-58b27b6f728c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.470786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fbaa1-0d1a-4815-8517-58b27b6f728c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.471067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fbaa1-0d1a-4815-8517-58b27b6f728c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.471872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fbaa1-0d1a-4815-8517-58b27b6f728c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:59.473207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fbaa1-0d1a-4815-8517-58b27b6f728c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-840fbaa1-0d1a-4815-8517-58b27b6f728c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:59.487274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:59.488342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:21:59.489256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 435/876] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:21:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:21:59.503317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fbaa1-0d1a-4815-8517-58b27b6f728c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:59.503518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:59.534731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89177c-4945-4a2b-843c-6fdc6a47d7be None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:59.582440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56804bd7-9d5c-4f3b-8695-ccd9e3a4d81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:59.583748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56804bd7-9d5c-4f3b-8695-ccd9e3a4d81c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.584108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56804bd7-9d5c-4f3b-8695-ccd9e3a4d81c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.584459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56804bd7-9d5c-4f3b-8695-ccd9e3a4d81c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.593789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56804bd7-9d5c-4f3b-8695-ccd9e3a4d81c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:59.595347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56804bd7-9d5c-4f3b-8695-ccd9e3a4d81c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:59.608632 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fbaa1-0d1a-4815-8517-58b27b6f728c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-56804bd7-9d5c-4f3b-8695-ccd9e3a4d81c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:21:59.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:21:59.613980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:21:59.614844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:21:59.614844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:21:59.614844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:21:59.614844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:21:59.614844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:21:59.614844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:21:59.614844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:21:59.614844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:21:59.614844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 442/877] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:21:59.621620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56804bd7-9d5c-4f3b-8695-ccd9e3a4d81c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:59.626948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:59.646569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cf6b8-0ef1-4308-959d-f03dc672fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:59.647642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cf6b8-0ef1-4308-959d-f03dc672fc43 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.648124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cf6b8-0ef1-4308-959d-f03dc672fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.648613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4cf6b8-0ef1-4308-959d-f03dc672fc43 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.649543 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cf6b8-0ef1-4308-959d-f03dc672fc43 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:59.650062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cf6b8-0ef1-4308-959d-f03dc672fc43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:59.651700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4cf6b8-0ef1-4308-959d-f03dc672fc43 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:59.657103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 436/878] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:21:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:59.668558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cf6b8-0ef1-4308-959d-f03dc672fc43 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:59.729281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56804bd7-9d5c-4f3b-8695-ccd9e3a4d81c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:59.758012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:21:59.765896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35678530-a6a8-4b40-adbc-b1f10a068578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:59.766866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35678530-a6a8-4b40-adbc-b1f10a068578 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.767417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35678530-a6a8-4b40-adbc-b1f10a068578 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.767888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35678530-a6a8-4b40-adbc-b1f10a068578 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.768882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35678530-a6a8-4b40-adbc-b1f10a068578 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:59.769384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35678530-a6a8-4b40-adbc-b1f10a068578 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:59.771012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35678530-a6a8-4b40-adbc-b1f10a068578 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:59.772687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4cf6b8-0ef1-4308-959d-f03dc672fc43 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:59.784543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 443/879] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:21:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:21:59.795373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35678530-a6a8-4b40-adbc-b1f10a068578 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:21:59.809023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e146305-e1f8-4d4a-84e6-63882ae31123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:59.809694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e146305-e1f8-4d4a-84e6-63882ae31123 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.809976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e146305-e1f8-4d4a-84e6-63882ae31123 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.810248 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e146305-e1f8-4d4a-84e6-63882ae31123 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.810956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e146305-e1f8-4d4a-84e6-63882ae31123 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:59.811263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e146305-e1f8-4d4a-84e6-63882ae31123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:59.812670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e146305-e1f8-4d4a-84e6-63882ae31123 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:59.817672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 437/880] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:21:59 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:21:59.908037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35678530-a6a8-4b40-adbc-b1f10a068578 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:21:59.946570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb3477-b8c7-4d27-a8e9-dbb1b4c237ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:21:59.947305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb3477-b8c7-4d27-a8e9-dbb1b4c237ef None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:21:59.947601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb3477-b8c7-4d27-a8e9-dbb1b4c237ef None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:21:59.947878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb3477-b8c7-4d27-a8e9-dbb1b4c237ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:21:59.948735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb3477-b8c7-4d27-a8e9-dbb1b4c237ef None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=2fcefa2d73a84415875357ef03e2c8da) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:21:59.949050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb3477-b8c7-4d27-a8e9-dbb1b4c237ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:21:59.950546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cb3477-b8c7-4d27-a8e9-dbb1b4c237ef None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:21:59.957718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 444/881] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:21:59 2022] GET /identity/v3/role_assignments?user.id=2fcefa2d73a84415875357ef03e2c8da&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:00.433929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cb9f5-d875-4933-970a-b6e805ca9ac3 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:00.434492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cb9f5-d875-4933-970a-b6e805ca9ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:00.434492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cb9f5-d875-4933-970a-b6e805ca9ac3 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:00.436136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 438/882] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:00.441582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb50cc4-cce5-4439-994e-0b72e5aee3d7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:00.441869 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb50cc4-cce5-4439-994e-0b72e5aee3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:00.442151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb50cc4-cce5-4439-994e-0b72e5aee3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:00.459489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4cb50cc4-cce5-4439-994e-0b72e5aee3d7 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:00.460678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4cb50cc4-cce5-4439-994e-0b72e5aee3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:00.511807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:00.530431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 445/883] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:00.553024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cb9f5-d875-4933-970a-b6e805ca9ac3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:00.599105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:00.657831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cb9f5-d875-4933-970a-b6e805ca9ac3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:00.692420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0c670-8c37-4f9f-828f-b112080e2426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:00.693580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0c670-8c37-4f9f-828f-b112080e2426 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:00.694061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0c670-8c37-4f9f-828f-b112080e2426 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:00.694545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0c670-8c37-4f9f-828f-b112080e2426 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:00.695423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0c670-8c37-4f9f-828f-b112080e2426 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:00.697231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0c670-8c37-4f9f-828f-b112080e2426 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e3d0c670-8c37-4f9f-828f-b112080e2426 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:00.709646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:00.710645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 02 03:22:00.711571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:00.714521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 439/884] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:22:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:00.720192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb50cc4-cce5-4439-994e-0b72e5aee3d7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:00.820320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb50cc4-cce5-4439-994e-0b72e5aee3d7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:00.855375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fa21e-cacf-4844-b1e8-e100191c16d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:00.856046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fa21e-cacf-4844-b1e8-e100191c16d2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:00.856329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fa21e-cacf-4844-b1e8-e100191c16d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:00.856603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fa21e-cacf-4844-b1e8-e100191c16d2 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:00.857488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fa21e-cacf-4844-b1e8-e100191c16d2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:00.857800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fa21e-cacf-4844-b1e8-e100191c16d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:00.859345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fa21e-cacf-4844-b1e8-e100191c16d2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:00.864588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 446/885] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:22:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:00.870841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0c670-8c37-4f9f-828f-b112080e2426 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:00.974909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0c670-8c37-4f9f-828f-b112080e2426 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:00.982406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:01.010717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1485840-8e7b-4494-b4c7-c6ba192c2bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:01.011418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1485840-8e7b-4494-b4c7-c6ba192c2bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.011702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1485840-8e7b-4494-b4c7-c6ba192c2bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.011975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1485840-8e7b-4494-b4c7-c6ba192c2bc6 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.012677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1485840-8e7b-4494-b4c7-c6ba192c2bc6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:01.012980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1485840-8e7b-4494-b4c7-c6ba192c2bc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:01.014351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1485840-8e7b-4494-b4c7-c6ba192c2bc6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:01.023647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 440/886] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:22:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:01.025651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fa21e-cacf-4844-b1e8-e100191c16d2 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:01.080523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:01.130587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fa21e-cacf-4844-b1e8-e100191c16d2 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:01.162596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ad2eb-1948-4f05-92e5-29c6788b4d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:01.163324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ad2eb-1948-4f05-92e5-29c6788b4d2e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.163860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ad2eb-1948-4f05-92e5-29c6788b4d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.164137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ad2eb-1948-4f05-92e5-29c6788b4d2e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.164992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ad2eb-1948-4f05-92e5-29c6788b4d2e None admin] RBAC: Adding query filter params (interface=public, service_id=86162c7b0ccf492b88f641dc79bec73e, region_id=RegionOne) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:01.165306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ad2eb-1948-4f05-92e5-29c6788b4d2e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:01.166688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ad2eb-1948-4f05-92e5-29c6788b4d2e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:01.174483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 447/887] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Jul 2 03:22:01 2022] GET /identity/v3/endpoints?service_id=86162c7b0ccf492b88f641dc79bec73e&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:01.180021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1485840-8e7b-4494-b4c7-c6ba192c2bc6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:01.285000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1485840-8e7b-4494-b4c7-c6ba192c2bc6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:01.287624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95aeef-a94d-4719-b573-b81e6f1d1c29 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.287915 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95aeef-a94d-4719-b573-b81e6f1d1c29 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.288184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95aeef-a94d-4719-b573-b81e6f1d1c29 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.290097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 448/888] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:01.295198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220067e-858b-4791-a2de-75d8aaab8b2b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.295607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220067e-858b-4791-a2de-75d8aaab8b2b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.296088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220067e-858b-4791-a2de-75d8aaab8b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.316894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2220067e-858b-4791-a2de-75d8aaab8b2b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:01.319197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2220067e-858b-4791-a2de-75d8aaab8b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:01.323324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6f88-5d35-4bdf-bbc2-27880ee4905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:01.324176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6f88-5d35-4bdf-bbc2-27880ee4905a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.324605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6f88-5d35-4bdf-bbc2-27880ee4905a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.325013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6f88-5d35-4bdf-bbc2-27880ee4905a None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.325823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d6f88-5d35-4bdf-bbc2-27880ee4905a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:01.327386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d6f88-5d35-4bdf-bbc2-27880ee4905a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:01.333989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 441/889] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Jul 2 03:22:01 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:22:01.405427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 449/890] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:01.428948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6f88-5d35-4bdf-bbc2-27880ee4905a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:01.465843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:01.538442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6f88-5d35-4bdf-bbc2-27880ee4905a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:01.573733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0399d9-1de9-486c-8acc-d4af3a01dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:01.574554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0399d9-1de9-486c-8acc-d4af3a01dd45 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.574831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0399d9-1de9-486c-8acc-d4af3a01dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.575104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0399d9-1de9-486c-8acc-d4af3a01dd45 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.575774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0399d9-1de9-486c-8acc-d4af3a01dd45 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:01.577150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0399d9-1de9-486c-8acc-d4af3a01dd45 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7a0399d9-1de9-486c-8acc-d4af3a01dd45 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:01.589882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:01.590853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:01.591756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 442/891] 10.222.0.42 () {56 vars in 1132 bytes} [Sat Jul 2 03:22:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:01.597584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220067e-858b-4791-a2de-75d8aaab8b2b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:01.644553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:01.702755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220067e-858b-4791-a2de-75d8aaab8b2b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:01.737644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4a4c5-2e32-4dae-91a0-ae6c78079468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:01.738313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4a4c5-2e32-4dae-91a0-ae6c78079468 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.738592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4a4c5-2e32-4dae-91a0-ae6c78079468 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.738871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4a4c5-2e32-4dae-91a0-ae6c78079468 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.739617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4a4c5-2e32-4dae-91a0-ae6c78079468 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:01.739924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4a4c5-2e32-4dae-91a0-ae6c78079468 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:01.741297 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4a4c5-2e32-4dae-91a0-ae6c78079468 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:01.746628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 450/892] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:22:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:01.752373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0399d9-1de9-486c-8acc-d4af3a01dd45 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:01.859745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:01.864210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0399d9-1de9-486c-8acc-d4af3a01dd45 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:01.900867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c24d8-d1ce-4657-9373-0ab43e41b3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:01.901930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c24d8-d1ce-4657-9373-0ab43e41b3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.902410 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c24d8-d1ce-4657-9373-0ab43e41b3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.902887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c24d8-d1ce-4657-9373-0ab43e41b3e2 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.903817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c24d8-d1ce-4657-9373-0ab43e41b3e2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:01.904321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c24d8-d1ce-4657-9373-0ab43e41b3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:01.905902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c24d8-d1ce-4657-9373-0ab43e41b3e2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:01.911755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 443/893] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:22:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:01.923328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4a4c5-2e32-4dae-91a0-ae6c78079468 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:01.958416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a3a1-d664-4526-aaf9-2ccb5531009a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.958738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a3a1-d664-4526-aaf9-2ccb5531009a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.958969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a3a1-d664-4526-aaf9-2ccb5531009a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.961732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 444/894] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:01.965892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ee8e2-9d0d-40eb-b541-e0d454a7283f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:01.966257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ee8e2-9d0d-40eb-b541-e0d454a7283f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:01.966550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ee8e2-9d0d-40eb-b541-e0d454a7283f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:01.984981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c93ee8e2-9d0d-40eb-b541-e0d454a7283f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:01.986158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c93ee8e2-9d0d-40eb-b541-e0d454a7283f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:02.023147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4a4c5-2e32-4dae-91a0-ae6c78079468 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:02.032102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:02.064597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030b9ca-0d8d-469a-8b88-e47057ae54e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:02.065546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030b9ca-0d8d-469a-8b88-e47057ae54e1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:02.066006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030b9ca-0d8d-469a-8b88-e47057ae54e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:02.066420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030b9ca-0d8d-469a-8b88-e47057ae54e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:02.067431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030b9ca-0d8d-469a-8b88-e47057ae54e1 None admin] RBAC: Adding query filter params (interface=public, service_id=292bf7a7885040188b772510582338e7, region_id=RegionOne) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:02.075322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 445/895] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:02.075506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030b9ca-0d8d-469a-8b88-e47057ae54e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:02.075506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030b9ca-0d8d-469a-8b88-e47057ae54e1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:02.078698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 451/896] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Jul 2 03:22:01 2022] GET /identity/v3/endpoints?service_id=292bf7a7885040188b772510582338e7&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:02.084128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ee8e2-9d0d-40eb-b541-e0d454a7283f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:02.098429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030b9ca-0d8d-469a-8b88-e47057ae54e1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:02.201420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:02.213857 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ee8e2-9d0d-40eb-b541-e0d454a7283f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:02.230961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030b9ca-0d8d-469a-8b88-e47057ae54e1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:02.251499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d71b35-39e9-4b5e-a3cd-d90ec1886b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:02.252255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d71b35-39e9-4b5e-a3cd-d90ec1886b40 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:02.252549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d71b35-39e9-4b5e-a3cd-d90ec1886b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:02.252831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d71b35-39e9-4b5e-a3cd-d90ec1886b40 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:02.253512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d71b35-39e9-4b5e-a3cd-d90ec1886b40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:02.254903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d71b35-39e9-4b5e-a3cd-d90ec1886b40 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:02.271328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 446/897] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Jul 2 03:22:02 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:22:02.271328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cf911-24e9-4bef-8fba-24dbce1d156a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:02.271328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cf911-24e9-4bef-8fba-24dbce1d156a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:02.271328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cf911-24e9-4bef-8fba-24dbce1d156a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:02.271328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 447/898] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:02.271328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158854e5-cbe6-4c3e-a234-eca18b11ef12 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:02.271328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158854e5-cbe6-4c3e-a234-eca18b11ef12 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:02.271993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158854e5-cbe6-4c3e-a234-eca18b11ef12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:02.278938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25524969-3557-4f00-9f9c-49c34065b0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:02.279813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25524969-3557-4f00-9f9c-49c34065b0ca None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:02.280166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25524969-3557-4f00-9f9c-49c34065b0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:02.280515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25524969-3557-4f00-9f9c-49c34065b0ca None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:02.281378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25524969-3557-4f00-9f9c-49c34065b0ca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:02.282862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25524969-3557-4f00-9f9c-49c34065b0ca None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:02.290168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-158854e5-cbe6-4c3e-a234-eca18b11ef12 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:02.291749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-158854e5-cbe6-4c3e-a234-eca18b11ef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:02.303501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 452/899] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:22:02 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:22:02.383151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 448/900] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:02.408188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25524969-3557-4f00-9f9c-49c34065b0ca None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:02.444345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:02.525073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25524969-3557-4f00-9f9c-49c34065b0ca None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:02.566559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b732d-2d9d-40f1-8559-47124cc34180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:02.567409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b732d-2d9d-40f1-8559-47124cc34180 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:02.567756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b732d-2d9d-40f1-8559-47124cc34180 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:02.568031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b732d-2d9d-40f1-8559-47124cc34180 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:02.579005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b732d-2d9d-40f1-8559-47124cc34180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a88b732d-2d9d-40f1-8559-47124cc34180 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:02.587561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:02.588497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:02.589353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 453/901] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Jul 2 03:22:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:02.595412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158854e5-cbe6-4c3e-a234-eca18b11ef12 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:02.687115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:02.706060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158854e5-cbe6-4c3e-a234-eca18b11ef12 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:02.740870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b84cdc-133a-4b5c-b234-41285af771e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:02.741594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b84cdc-133a-4b5c-b234-41285af771e9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:02.741877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b84cdc-133a-4b5c-b234-41285af771e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:02.742151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b84cdc-133a-4b5c-b234-41285af771e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:02.743092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b84cdc-133a-4b5c-b234-41285af771e9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:02.743466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b84cdc-133a-4b5c-b234-41285af771e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:02.744851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b84cdc-133a-4b5c-b234-41285af771e9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:02.753861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 449/902] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Jul 2 03:22:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:02.759287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b732d-2d9d-40f1-8559-47124cc34180 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:02.853318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:02.863886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b732d-2d9d-40f1-8559-47124cc34180 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:02.898921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b6b90-17ac-4e02-b8f1-67c45dc1f0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:02.899765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b6b90-17ac-4e02-b8f1-67c45dc1f0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:02.900049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b6b90-17ac-4e02-b8f1-67c45dc1f0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:02.900322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b6b90-17ac-4e02-b8f1-67c45dc1f0c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:02.911226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b6b90-17ac-4e02-b8f1-67c45dc1f0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:02.912764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b6b90-17ac-4e02-b8f1-67c45dc1f0c7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-e26b6b90-17ac-4e02-b8f1-67c45dc1f0c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:02.934608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:02.935556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:02.936438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:02.936438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:02.936438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:02.936438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:02.936438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 454/903] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:02.941901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b84cdc-133a-4b5c-b234-41285af771e9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:03.047724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b84cdc-133a-4b5c-b234-41285af771e9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:03.089393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdccdc-16a7-4522-8f4f-781e1e6d98fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:03.090060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdccdc-16a7-4522-8f4f-781e1e6d98fd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:03.090340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdccdc-16a7-4522-8f4f-781e1e6d98fd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:03.090617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdccdc-16a7-4522-8f4f-781e1e6d98fd None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:03.091361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdccdc-16a7-4522-8f4f-781e1e6d98fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:03.091672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdccdc-16a7-4522-8f4f-781e1e6d98fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:03.093059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdccdc-16a7-4522-8f4f-781e1e6d98fd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:03.106315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 450/904] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:03.111559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b6b90-17ac-4e02-b8f1-67c45dc1f0c7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:03.218621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b6b90-17ac-4e02-b8f1-67c45dc1f0c7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:03.253224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d3975-434b-46f2-ab29-a43306af0b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:03.254033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d3975-434b-46f2-ab29-a43306af0b94 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:03.254317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d3975-434b-46f2-ab29-a43306af0b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:03.254592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762d3975-434b-46f2-ab29-a43306af0b94 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:03.262432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d3975-434b-46f2-ab29-a43306af0b94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:03.263919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762d3975-434b-46f2-ab29-a43306af0b94 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-762d3975-434b-46f2-ab29-a43306af0b94 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:22:03.278832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:03.280966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:03.283154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 455/905] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:22:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:03.287331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdccdc-16a7-4522-8f4f-781e1e6d98fd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:03.291002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:03.391798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdccdc-16a7-4522-8f4f-781e1e6d98fd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:03.427353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6cd229-eecd-4445-9613-6ed25086334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:03.428030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cd229-eecd-4445-9613-6ed25086334c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:03.428310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cd229-eecd-4445-9613-6ed25086334c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:03.428583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6cd229-eecd-4445-9613-6ed25086334c None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:03.429422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cd229-eecd-4445-9613-6ed25086334c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:03.429728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cd229-eecd-4445-9613-6ed25086334c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:03.431837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6cd229-eecd-4445-9613-6ed25086334c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:03.450329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 451/906] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Jul 2 03:22:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:03.456023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d3975-434b-46f2-ab29-a43306af0b94 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:03.459263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:03.523495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24aeb9e-e93a-492c-820d-61e2cd3b87e1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:03.523702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24aeb9e-e93a-492c-820d-61e2cd3b87e1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:03.524010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24aeb9e-e93a-492c-820d-61e2cd3b87e1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:03.525618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 452/907] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:03.530849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cb994-42bf-4aff-bd97-a2747df32b5d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:03.531229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cb994-42bf-4aff-bd97-a2747df32b5d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:03.531479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cb994-42bf-4aff-bd97-a2747df32b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:03.549194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b36cb994-42bf-4aff-bd97-a2747df32b5d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:03.550666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b36cb994-42bf-4aff-bd97-a2747df32b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:03.571044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762d3975-434b-46f2-ab29-a43306af0b94 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:03.606693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d7c97-2362-4483-a4b4-e0dd21d569c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:03.607774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d7c97-2362-4483-a4b4-e0dd21d569c3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:03.608263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d7c97-2362-4483-a4b4-e0dd21d569c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:03.608753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d7c97-2362-4483-a4b4-e0dd21d569c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:03.621065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:03.627561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 453/908] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:03.642955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cb994-42bf-4aff-bd97-a2747df32b5d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:03.680450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d7c97-2362-4483-a4b4-e0dd21d569c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:03.682586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d7c97-2362-4483-a4b4-e0dd21d569c3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:03.684704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:03.741134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cb994-42bf-4aff-bd97-a2747df32b5d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:03.774814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23b3e4-1faa-4e4a-a479-d6d06b900ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:03.775576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23b3e4-1faa-4e4a-a479-d6d06b900ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:03.775858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23b3e4-1faa-4e4a-a479-d6d06b900ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:03.776129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23b3e4-1faa-4e4a-a479-d6d06b900ef8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:03.776781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23b3e4-1faa-4e4a-a479-d6d06b900ef8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:03.778178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23b3e4-1faa-4e4a-a479-d6d06b900ef8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0e23b3e4-1faa-4e4a-a479-d6d06b900ef8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:03.791616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:03.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 02 03:22:03.793602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:03.794855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 456/909] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:03 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:22:03.796377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d7c97-2362-4483-a4b4-e0dd21d569c3 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:03.796668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 454/910] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:22:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:03.797981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23b3e4-1faa-4e4a-a479-d6d06b900ef8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:03.897823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:03.904840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d7c97-2362-4483-a4b4-e0dd21d569c3 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:03.906662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:03.907810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23b3e4-1faa-4e4a-a479-d6d06b900ef8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:03.943680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9013-050d-4b75-b1b3-ebf9cee2c0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:03.943899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacc6c0-8dd0-44e6-9e38-f11b1d96ae06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:03.944510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9013-050d-4b75-b1b3-ebf9cee2c0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:03.944855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9013-050d-4b75-b1b3-ebf9cee2c0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:03.945200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd9013-050d-4b75-b1b3-ebf9cee2c0a3 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:03.945512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacc6c0-8dd0-44e6-9e38-f11b1d96ae06 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:03.946109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacc6c0-8dd0-44e6-9e38-f11b1d96ae06 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:03.946306 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd9013-050d-4b75-b1b3-ebf9cee2c0a3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:03.946432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd9013-050d-4b75-b1b3-ebf9cee2c0a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:03.946890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacc6c0-8dd0-44e6-9e38-f11b1d96ae06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:03.948095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd9013-050d-4b75-b1b3-ebf9cee2c0a3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:03.953768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 455/911] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:22:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:03.959244 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9013-050d-4b75-b1b3-ebf9cee2c0a3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:03.959811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacc6c0-8dd0-44e6-9e38-f11b1d96ae06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:03.961528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacc6c0-8dd0-44e6-9e38-f11b1d96ae06 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1bacc6c0-8dd0-44e6-9e38-f11b1d96ae06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:03.980192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:03.981218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:03.982111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:03.982111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:03.982111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:03.982111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:03.988791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 457/912] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:03.989906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacc6c0-8dd0-44e6-9e38-f11b1d96ae06 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:04.061092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd9013-050d-4b75-b1b3-ebf9cee2c0a3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:04.091690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:04.095413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacc6c0-8dd0-44e6-9e38-f11b1d96ae06 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:04.096567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c46a52-7026-40b4-8045-649d324173d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:04.097542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c46a52-7026-40b4-8045-649d324173d0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.097945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c46a52-7026-40b4-8045-649d324173d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.098321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c46a52-7026-40b4-8045-649d324173d0 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.099904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c46a52-7026-40b4-8045-649d324173d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:04.100221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c46a52-7026-40b4-8045-649d324173d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:04.101619 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c46a52-7026-40b4-8045-649d324173d0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:04.118040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 456/913] 10.222.0.42 () {56 vars in 1114 bytes} [Sat Jul 2 03:22:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:04.119245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c46a52-7026-40b4-8045-649d324173d0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:04.151790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48f40e-af2d-4706-9cba-b63972b18631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:04.153124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48f40e-af2d-4706-9cba-b63972b18631 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.153885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48f40e-af2d-4706-9cba-b63972b18631 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.154758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48f40e-af2d-4706-9cba-b63972b18631 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.155967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48f40e-af2d-4706-9cba-b63972b18631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:04.156695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48f40e-af2d-4706-9cba-b63972b18631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:04.158965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a48f40e-af2d-4706-9cba-b63972b18631 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:04.174675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 458/914] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:04.179970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48f40e-af2d-4706-9cba-b63972b18631 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:04.202954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:04.247720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c46a52-7026-40b4-8045-649d324173d0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:04.285549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fed59-14a9-41fd-954a-54f271d34954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:04.286503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fed59-14a9-41fd-954a-54f271d34954 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.286948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fed59-14a9-41fd-954a-54f271d34954 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.287422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fed59-14a9-41fd-954a-54f271d34954 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.288319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fed59-14a9-41fd-954a-54f271d34954 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:04.289990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fed59-14a9-41fd-954a-54f271d34954 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:04.316386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 457/915] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Jul 2 03:22:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:04.319848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48f40e-af2d-4706-9cba-b63972b18631 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:04.369938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2623a-f3d0-4d16-8928-133452c71728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:04.371272 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2623a-f3d0-4d16-8928-133452c71728 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.371861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2623a-f3d0-4d16-8928-133452c71728 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.372358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2623a-f3d0-4d16-8928-133452c71728 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.381019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2623a-f3d0-4d16-8928-133452c71728 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:04.382929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a2623a-f3d0-4d16-8928-133452c71728 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c3a2623a-f3d0-4d16-8928-133452c71728 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:22:04.399573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:04.400634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:04.401615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:22:04.401615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:22:04.401615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:22:04.401615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:22:04.401615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:22:04.401615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:22:04.401615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:04.401615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:04.404546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 459/916] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:04.410623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fed59-14a9-41fd-954a-54f271d34954 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:04.433287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86566148-80d1-4ef7-be3b-7dcfb4a2c27a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.433632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86566148-80d1-4ef7-be3b-7dcfb4a2c27a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.433844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86566148-80d1-4ef7-be3b-7dcfb4a2c27a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.435539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 460/917] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:04.441033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ea7c-2e8b-4024-8894-5a847007f182 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.441676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ea7c-2e8b-4024-8894-5a847007f182 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.442347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4ea7c-2e8b-4024-8894-5a847007f182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.461579 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8cd4ea7c-2e8b-4024-8894-5a847007f182 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:04.462803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8cd4ea7c-2e8b-4024-8894-5a847007f182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:04.464411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:04.544112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fed59-14a9-41fd-954a-54f271d34954 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:04.554633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:04.563461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 461/918] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:04.563461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c6684-574b-47b6-808c-7d5ec407ec59 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.563461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c6684-574b-47b6-808c-7d5ec407ec59 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.563461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c6684-574b-47b6-808c-7d5ec407ec59 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.563866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 462/919] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:04.576841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cf122-d807-4f14-8c23-be36bd2c77aa None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.577186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cf122-d807-4f14-8c23-be36bd2c77aa None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.577657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cf122-d807-4f14-8c23-be36bd2c77aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.586456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f9474-0f82-4b3d-8a6b-784f50f8cf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:04.587126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f9474-0f82-4b3d-8a6b-784f50f8cf00 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.587444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f9474-0f82-4b3d-8a6b-784f50f8cf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.587722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f9474-0f82-4b3d-8a6b-784f50f8cf00 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.588515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f9474-0f82-4b3d-8a6b-784f50f8cf00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:04.588829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f9474-0f82-4b3d-8a6b-784f50f8cf00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:04.590304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f9474-0f82-4b3d-8a6b-784f50f8cf00 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:04.596808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bd1cf122-d807-4f14-8c23-be36bd2c77aa None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:04.597991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bd1cf122-d807-4f14-8c23-be36bd2c77aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:04.604173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 458/920] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:22:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:04.605251 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f9474-0f82-4b3d-8a6b-784f50f8cf00 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:04.667746 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 463/921] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:04.668608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cf122-d807-4f14-8c23-be36bd2c77aa None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:04.716973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:04.720363 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f9474-0f82-4b3d-8a6b-784f50f8cf00 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:04.758168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962c016-68b6-4228-aab0-a0f744b178de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:04.759124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962c016-68b6-4228-aab0-a0f744b178de None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.759440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962c016-68b6-4228-aab0-a0f744b178de None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.759721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962c016-68b6-4228-aab0-a0f744b178de None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.760373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962c016-68b6-4228-aab0-a0f744b178de None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:04.761778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2962c016-68b6-4228-aab0-a0f744b178de None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2962c016-68b6-4228-aab0-a0f744b178de None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:04.773968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:04.774963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 02 03:22:04.781078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:04.781862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cf122-d807-4f14-8c23-be36bd2c77aa None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:04.781944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 459/922] 10.222.0.42 () {56 vars in 1132 bytes} [Sat Jul 2 03:22:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:04.781944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962c016-68b6-4228-aab0-a0f744b178de None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:04.811472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:04.822547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c4450-4fe8-4269-b73d-ca4129b19634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:04.823217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c4450-4fe8-4269-b73d-ca4129b19634 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.823889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c4450-4fe8-4269-b73d-ca4129b19634 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.824225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c4450-4fe8-4269-b73d-ca4129b19634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.825382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c4450-4fe8-4269-b73d-ca4129b19634 None admin] RBAC: Adding query filter params (role.id=d5b0722ac98a4453bc506335e1e71cab, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=2fcefa2d73a84415875357ef03e2c8da) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:04.825865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c4450-4fe8-4269-b73d-ca4129b19634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:04.827419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c4450-4fe8-4269-b73d-ca4129b19634 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:04.833237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 464/923] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:04 2022] GET /identity/v3/role_assignments?user.id=2fcefa2d73a84415875357ef03e2c8da&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5b0722ac98a4453bc506335e1e71cab => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:04.834047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c4450-4fe8-4269-b73d-ca4129b19634 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:04.889912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2962c016-68b6-4228-aab0-a0f744b178de None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:04.930147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a780e22-9429-4b24-9e41-c3f0968a47dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:04.931344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a780e22-9429-4b24-9e41-c3f0968a47dd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:04.932314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a780e22-9429-4b24-9e41-c3f0968a47dd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:04.933094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a780e22-9429-4b24-9e41-c3f0968a47dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:04.934304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a780e22-9429-4b24-9e41-c3f0968a47dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:04.935939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a780e22-9429-4b24-9e41-c3f0968a47dd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9a780e22-9429-4b24-9e41-c3f0968a47dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:04.949868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:04.951022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:04.951896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:04.954286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 460/924] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:22:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:04.960552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a780e22-9429-4b24-9e41-c3f0968a47dd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:04.977944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c4450-4fe8-4269-b73d-ca4129b19634 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:04.994264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:05.029638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b70f44-d848-4d6e-945f-fcc5bf74c9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:05.030311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b70f44-d848-4d6e-945f-fcc5bf74c9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:05.030596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b70f44-d848-4d6e-945f-fcc5bf74c9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:05.030873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b70f44-d848-4d6e-945f-fcc5bf74c9e1 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:05.031618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b70f44-d848-4d6e-945f-fcc5bf74c9e1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:05.031928 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b70f44-d848-4d6e-945f-fcc5bf74c9e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:05.033321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b70f44-d848-4d6e-945f-fcc5bf74c9e1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:05.043174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 465/925] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:22:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:05.045109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b70f44-d848-4d6e-945f-fcc5bf74c9e1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:05.093088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a780e22-9429-4b24-9e41-c3f0968a47dd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:05.126333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:05.131223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee5792-19dc-4cc1-8648-0556f1c6f2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:05.132035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee5792-19dc-4cc1-8648-0556f1c6f2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:05.132325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee5792-19dc-4cc1-8648-0556f1c6f2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:05.132608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee5792-19dc-4cc1-8648-0556f1c6f2b8 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:05.133375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee5792-19dc-4cc1-8648-0556f1c6f2b8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:05.133682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee5792-19dc-4cc1-8648-0556f1c6f2b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:05.135068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee5792-19dc-4cc1-8648-0556f1c6f2b8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:05.144778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 461/926] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:22:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:05.145901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee5792-19dc-4cc1-8648-0556f1c6f2b8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:05.200883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b70f44-d848-4d6e-945f-fcc5bf74c9e1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:05.238613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18c398-2e69-499d-9ed6-b35f096bed8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:05.239374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18c398-2e69-499d-9ed6-b35f096bed8e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:05.239617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18c398-2e69-499d-9ed6-b35f096bed8e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:05.239898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18c398-2e69-499d-9ed6-b35f096bed8e None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:05.240605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18c398-2e69-499d-9ed6-b35f096bed8e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:05.240911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18c398-2e69-499d-9ed6-b35f096bed8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:05.242360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18c398-2e69-499d-9ed6-b35f096bed8e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:05.252427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 466/927] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:22:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:05.254622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18c398-2e69-499d-9ed6-b35f096bed8e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:05.257018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:05.267478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee5792-19dc-4cc1-8648-0556f1c6f2b8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:05.302860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25e9ef-52f1-4aa7-98e7-6ef792d91b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:05.303948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25e9ef-52f1-4aa7-98e7-6ef792d91b83 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:05.304087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25e9ef-52f1-4aa7-98e7-6ef792d91b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:05.304475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25e9ef-52f1-4aa7-98e7-6ef792d91b83 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:05.305465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25e9ef-52f1-4aa7-98e7-6ef792d91b83 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:05.305877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25e9ef-52f1-4aa7-98e7-6ef792d91b83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:05.307722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25e9ef-52f1-4aa7-98e7-6ef792d91b83 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:05.318026 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 462/928] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:22:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:05.319020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25e9ef-52f1-4aa7-98e7-6ef792d91b83 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:05.390092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18c398-2e69-499d-9ed6-b35f096bed8e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:05.419846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:05.425566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25e9ef-52f1-4aa7-98e7-6ef792d91b83 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:05.436641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b4f97-ad10-4512-ae43-51c306f7fd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:05.437520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b4f97-ad10-4512-ae43-51c306f7fd2a None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:05.438346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b4f97-ad10-4512-ae43-51c306f7fd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:05.438798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b4f97-ad10-4512-ae43-51c306f7fd2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:05.439966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b4f97-ad10-4512-ae43-51c306f7fd2a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:05.442226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b4f97-ad10-4512-ae43-51c306f7fd2a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:05.465366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b386b-323a-4b22-b041-472c3fae1066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:05.466192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b386b-323a-4b22-b041-472c3fae1066 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:05.466590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b386b-323a-4b22-b041-472c3fae1066 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:05.466965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b386b-323a-4b22-b041-472c3fae1066 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:05.467932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b386b-323a-4b22-b041-472c3fae1066 None admin] RBAC: Adding query filter params (interface=public, service_id=cd34c316a773474097c8922b96188a62, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:05.475342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b386b-323a-4b22-b041-472c3fae1066 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:05.475342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b386b-323a-4b22-b041-472c3fae1066 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:05.476040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 467/929] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Jul 2 03:22:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:05.476658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 463/930] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Jul 2 03:22:05 2022] GET /identity/v3/endpoints?service_id=cd34c316a773474097c8922b96188a62&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:05.482514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b4f97-ad10-4512-ae43-51c306f7fd2a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:05.609397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b4f97-ad10-4512-ae43-51c306f7fd2a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:05.646306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d8612-ace0-44d8-b7ed-999eff32a2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:05.646668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d8612-ace0-44d8-b7ed-999eff32a2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:05.646899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d8612-ace0-44d8-b7ed-999eff32a2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:05.647177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d8612-ace0-44d8-b7ed-999eff32a2e8 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:05.647892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d8612-ace0-44d8-b7ed-999eff32a2e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:05.649382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d8612-ace0-44d8-b7ed-999eff32a2e8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:05.656364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 468/931] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Jul 2 03:22:05 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:22:06.428935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1bc3c-efb4-417b-ac6a-b487201b4026 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:06.429460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1bc3c-efb4-417b-ac6a-b487201b4026 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:06.429460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1bc3c-efb4-417b-ac6a-b487201b4026 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:06.431058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 464/932] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:06.437401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d41c7-814c-4e74-8d4e-461e4693bd80 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:06.437401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d41c7-814c-4e74-8d4e-461e4693bd80 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:06.437401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d41c7-814c-4e74-8d4e-461e4693bd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:06.454745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1c1d41c7-814c-4e74-8d4e-461e4693bd80 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:06.456247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1c1d41c7-814c-4e74-8d4e-461e4693bd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:06.488183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:06.526787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 469/933] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:06.549859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1bc3c-efb4-417b-ac6a-b487201b4026 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:06.573515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:06.655607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1bc3c-efb4-417b-ac6a-b487201b4026 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:06.694611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737107b-1dbb-4a58-bc5f-ef110288a1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:06.695584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737107b-1dbb-4a58-bc5f-ef110288a1fd None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:06.696599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737107b-1dbb-4a58-bc5f-ef110288a1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:06.697042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737107b-1dbb-4a58-bc5f-ef110288a1fd None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:06.697965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737107b-1dbb-4a58-bc5f-ef110288a1fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:06.699600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737107b-1dbb-4a58-bc5f-ef110288a1fd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:06.701181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING keystone.server.flask.common [None req-2737107b-1dbb-4a58-bc5f-ef110288a1fd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 02 03:22:06.721778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.sql.core [None req-2737107b-1dbb-4a58-bc5f-ef110288a1fd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 02 03:22:06.721778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 02 03:22:06.721778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [parameters: {'id': 'f3b9dc5b4d5e41f4a0b82fe8e3f38c9c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 02 03:22:06.721778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80353) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2737107b-1dbb-4a58-bc5f-ef110288a1fd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f3b9dc5b4d5e41f4a0b82fe8e3f38c9c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f3b9dc5b4d5e41f4a0b82fe8e3f38c9c. Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 02 03:22:06.740332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 02 03:22:06.741382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application yield resource Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.rollback() Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.force_reraise() Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise self.value Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application session.commit() Jul 02 03:22:06.742631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._prepare_impl() Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.flush() Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._flush(objects) Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application compat.raise_( Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application flush_context.execute() Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 02 03:22:06.744559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rec.execute(self) Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 02 03:22:06.745667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 02 03:22:06.746579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [parameters: {'id': 'f3b9dc5b4d5e41f4a0b82fe8e3f38c9c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:06.747481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:06.748284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:06.749239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:06.749239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 02 03:22:06.749239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 02 03:22:06.749239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:06.749239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:06.749239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 02 03:22:06.749239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 02 03:22:06.749239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f3b9dc5b4d5e41f4a0b82fe8e3f38c9c. Jul 02 03:22:06.749239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:06.755689 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 465/934] 10.222.0.42 () {60 vars in 1140 bytes} [Sat Jul 2 03:22:06 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:06.758109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d41c7-814c-4e74-8d4e-461e4693bd80 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:06.850993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e55ebe-7885-4937-ad8d-77083fff5f75 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:06.851375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e55ebe-7885-4937-ad8d-77083fff5f75 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:06.851574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e55ebe-7885-4937-ad8d-77083fff5f75 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:06.859785 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 466/935] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:06.869273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61ffe-4e99-4755-ae56-b15e41786a01 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:06.869504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d41c7-814c-4e74-8d4e-461e4693bd80 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:06.869766 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61ffe-4e99-4755-ae56-b15e41786a01 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:06.870150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd61ffe-4e99-4755-ae56-b15e41786a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:06.889417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2bd61ffe-4e99-4755-ae56-b15e41786a01 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:06.891067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2bd61ffe-4e99-4755-ae56-b15e41786a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:06.905847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be1fcf-0198-4e43-84d4-6291e7c9b3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:06.906856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be1fcf-0198-4e43-84d4-6291e7c9b3ef None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:06.907307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be1fcf-0198-4e43-84d4-6291e7c9b3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:06.907744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be1fcf-0198-4e43-84d4-6291e7c9b3ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:06.914752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be1fcf-0198-4e43-84d4-6291e7c9b3ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:06.916554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be1fcf-0198-4e43-84d4-6291e7c9b3ef None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d9be1fcf-0198-4e43-84d4-6291e7c9b3ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:22:06.930724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:06.931993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:06.932879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:22:06.932879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:22:06.932879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:22:06.932879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:22:06.932879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:22:06.932879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:22:06.932879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:06.932879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:06.935430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 470/936] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:06.942218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be1fcf-0198-4e43-84d4-6291e7c9b3ef None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:06.979393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 467/937] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:06.982851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:06.995787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61ffe-4e99-4755-ae56-b15e41786a01 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:07.004927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:07.069488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be1fcf-0198-4e43-84d4-6291e7c9b3ef None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:07.111639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c923d40-692a-4d65-88e7-b6eb0fd1270f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:07.112836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c923d40-692a-4d65-88e7-b6eb0fd1270f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.113211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c923d40-692a-4d65-88e7-b6eb0fd1270f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.113578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c923d40-692a-4d65-88e7-b6eb0fd1270f None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.114470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c923d40-692a-4d65-88e7-b6eb0fd1270f None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:07.114914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c923d40-692a-4d65-88e7-b6eb0fd1270f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:07.117036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c923d40-692a-4d65-88e7-b6eb0fd1270f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:07.134871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd61ffe-4e99-4755-ae56-b15e41786a01 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:07.143384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 471/938] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:22:06 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:07.178308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156db9-48a3-4c26-bc6e-6b72d0471345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:07.179580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156db9-48a3-4c26-bc6e-6b72d0471345 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.180087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156db9-48a3-4c26-bc6e-6b72d0471345 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.180580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156db9-48a3-4c26-bc6e-6b72d0471345 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.193872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55156db9-48a3-4c26-bc6e-6b72d0471345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-55156db9-48a3-4c26-bc6e-6b72d0471345 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:07.202910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:07.203994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:07.204874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:07.207582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 468/939] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Jul 2 03:22:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:07.216348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c923d40-692a-4d65-88e7-b6eb0fd1270f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:07.317869 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:07.333709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c923d40-692a-4d65-88e7-b6eb0fd1270f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:07.376717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b6764-1de5-4879-aabd-f748545a2bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:07.377388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b6764-1de5-4879-aabd-f748545a2bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.377669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b6764-1de5-4879-aabd-f748545a2bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.377940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b6764-1de5-4879-aabd-f748545a2bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.378692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b6764-1de5-4879-aabd-f748545a2bc1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:07.378988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b6764-1de5-4879-aabd-f748545a2bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:07.380397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b6764-1de5-4879-aabd-f748545a2bc1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:07.391380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 472/940] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Jul 2 03:22:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:07.396021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156db9-48a3-4c26-bc6e-6b72d0471345 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:07.466043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:07.502699 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156db9-48a3-4c26-bc6e-6b72d0471345 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:07.536550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70f1a0-a2a1-46ab-a2ca-e9973363e80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:07.537365 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f1a0-a2a1-46ab-a2ca-e9973363e80a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.537715 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f1a0-a2a1-46ab-a2ca-e9973363e80a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.537974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f1a0-a2a1-46ab-a2ca-e9973363e80a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.548789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70f1a0-a2a1-46ab-a2ca-e9973363e80a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:07.550380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70f1a0-a2a1-46ab-a2ca-e9973363e80a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7d70f1a0-a2a1-46ab-a2ca-e9973363e80a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:07.568210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:07.569143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:07.570048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:07.570048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:07.570048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:07.570048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:07.574621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 469/941] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:07.576075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b6764-1de5-4879-aabd-f748545a2bc1 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:07.612154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae147c6-ab20-4c12-a68d-d9a116de91d6 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.612451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae147c6-ab20-4c12-a68d-d9a116de91d6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.612746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae147c6-ab20-4c12-a68d-d9a116de91d6 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.614265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 470/942] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:07.619184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81000348-f519-442e-ae58-afce98bd77f0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.619482 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81000348-f519-442e-ae58-afce98bd77f0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.619772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81000348-f519-442e-ae58-afce98bd77f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.636671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-81000348-f519-442e-ae58-afce98bd77f0 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:07.637868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-81000348-f519-442e-ae58-afce98bd77f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:07.698198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b6764-1de5-4879-aabd-f748545a2bc1 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:07.719554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 471/943] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:07.726400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1f824-7ad4-4a78-b7b6-d98c046a7ddd None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.726754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1f824-7ad4-4a78-b7b6-d98c046a7ddd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.727109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1f824-7ad4-4a78-b7b6-d98c046a7ddd None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.728925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 472/944] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:07.735197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1f824-7ad4-4a78-b7b6-d98c046a7ddd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:07.737950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f527f2-f92d-478d-bfc2-a34c3fef598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:07.738617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f527f2-f92d-478d-bfc2-a34c3fef598b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.738902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f527f2-f92d-478d-bfc2-a34c3fef598b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.739175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f527f2-f92d-478d-bfc2-a34c3fef598b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.739943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f527f2-f92d-478d-bfc2-a34c3fef598b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:07.740258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f527f2-f92d-478d-bfc2-a34c3fef598b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:07.741624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f527f2-f92d-478d-bfc2-a34c3fef598b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:07.753851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 473/945] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:07.755122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3f0cb-bb13-46c3-93ea-1f0ee8ae8552 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.755438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3f0cb-bb13-46c3-93ea-1f0ee8ae8552 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.755714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3f0cb-bb13-46c3-93ea-1f0ee8ae8552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.772737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5fc3f0cb-bb13-46c3-93ea-1f0ee8ae8552 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:07.773952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5fc3f0cb-bb13-46c3-93ea-1f0ee8ae8552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:07.777702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:07.793537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:07.846748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 474/946] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:07.847846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3f0cb-bb13-46c3-93ea-1f0ee8ae8552 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:07.852021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1f824-7ad4-4a78-b7b6-d98c046a7ddd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:07.895946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2392672-ab03-4987-86a3-8896fb98713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:07.896744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2392672-ab03-4987-86a3-8896fb98713b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:07.897019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2392672-ab03-4987-86a3-8896fb98713b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:07.897311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2392672-ab03-4987-86a3-8896fb98713b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:07.897966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2392672-ab03-4987-86a3-8896fb98713b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:07.899356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2392672-ab03-4987-86a3-8896fb98713b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a2392672-ab03-4987-86a3-8896fb98713b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:07.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:07.911971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:07.912803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 473/947] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:07.913589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2392672-ab03-4987-86a3-8896fb98713b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:07.964302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3f0cb-bb13-46c3-93ea-1f0ee8ae8552 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.000015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659bba7-f184-4eae-88a5-1c0825878169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.000820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659bba7-f184-4eae-88a5-1c0825878169 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.001102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659bba7-f184-4eae-88a5-1c0825878169 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.001374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659bba7-f184-4eae-88a5-1c0825878169 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.008613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659bba7-f184-4eae-88a5-1c0825878169 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.010053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659bba7-f184-4eae-88a5-1c0825878169 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.012761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:08.024746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2392672-ab03-4987-86a3-8896fb98713b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0659bba7-f184-4eae-88a5-1c0825878169 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:22:08.026385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:08.027352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:08.028302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 475/948] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:22:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:08.028933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659bba7-f184-4eae-88a5-1c0825878169 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.042419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:08.066994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc578afb-0f6b-48a2-bfd0-06237371f3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.067817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578afb-0f6b-48a2-bfd0-06237371f3cb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.068116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578afb-0f6b-48a2-bfd0-06237371f3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.068376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578afb-0f6b-48a2-bfd0-06237371f3cb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.069019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc578afb-0f6b-48a2-bfd0-06237371f3cb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.070387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc578afb-0f6b-48a2-bfd0-06237371f3cb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-fc578afb-0f6b-48a2-bfd0-06237371f3cb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:08.082337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:08.083597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:08.088269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 474/949] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:22:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:08.089063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc578afb-0f6b-48a2-bfd0-06237371f3cb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.145572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659bba7-f184-4eae-88a5-1c0825878169 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.180112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8767-5347-42ab-8e28-65f880b910bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.180768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8767-5347-42ab-8e28-65f880b910bb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.181045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8767-5347-42ab-8e28-65f880b910bb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.181315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8767-5347-42ab-8e28-65f880b910bb None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.182327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8767-5347-42ab-8e28-65f880b910bb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:08.182633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8767-5347-42ab-8e28-65f880b910bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.184032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8767-5347-42ab-8e28-65f880b910bb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.188858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 476/950] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:22:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:08.189455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8767-5347-42ab-8e28-65f880b910bb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.201908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc578afb-0f6b-48a2-bfd0-06237371f3cb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.238968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621ed54-47cf-4ee1-a40f-2c9701be7373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.239666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621ed54-47cf-4ee1-a40f-2c9701be7373 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.239947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621ed54-47cf-4ee1-a40f-2c9701be7373 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.240219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621ed54-47cf-4ee1-a40f-2c9701be7373 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.241424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621ed54-47cf-4ee1-a40f-2c9701be7373 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:08.241756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621ed54-47cf-4ee1-a40f-2c9701be7373 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.243525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621ed54-47cf-4ee1-a40f-2c9701be7373 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.250298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:08.270116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 475/951] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Jul 2 03:22:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:08.270116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621ed54-47cf-4ee1-a40f-2c9701be7373 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.286437 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8767-5347-42ab-8e28-65f880b910bb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.309576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:08.324210 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ac19f-a659-4d73-945e-fbfaf40d4892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.324961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ac19f-a659-4d73-945e-fbfaf40d4892 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.325171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ac19f-a659-4d73-945e-fbfaf40d4892 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.325449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ac19f-a659-4d73-945e-fbfaf40d4892 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.326162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ac19f-a659-4d73-945e-fbfaf40d4892 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:08.326472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ac19f-a659-4d73-945e-fbfaf40d4892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.328012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ac19f-a659-4d73-945e-fbfaf40d4892 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.333027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 477/952] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:22:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:08.333809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ac19f-a659-4d73-945e-fbfaf40d4892 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.381592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621ed54-47cf-4ee1-a40f-2c9701be7373 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.418296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d025d13-5498-413d-b499-f252e7b136fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.418959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d025d13-5498-413d-b499-f252e7b136fa None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.419245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d025d13-5498-413d-b499-f252e7b136fa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.419499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d025d13-5498-413d-b499-f252e7b136fa None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.420649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d025d13-5498-413d-b499-f252e7b136fa None admin] RBAC: Adding query filter params (type=compute) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:08.420953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d025d13-5498-413d-b499-f252e7b136fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.422675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d025d13-5498-413d-b499-f252e7b136fa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.439332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 476/953] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:22:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:08.439332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d025d13-5498-413d-b499-f252e7b136fa None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.460583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ac19f-a659-4d73-945e-fbfaf40d4892 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.499756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:08.510845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabacba2-a336-4ca5-83ca-4f9de2ff41a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.511863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabacba2-a336-4ca5-83ca-4f9de2ff41a5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.512260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabacba2-a336-4ca5-83ca-4f9de2ff41a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.512625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabacba2-a336-4ca5-83ca-4f9de2ff41a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.527775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabacba2-a336-4ca5-83ca-4f9de2ff41a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.529331 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabacba2-a336-4ca5-83ca-4f9de2ff41a5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.544887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-dabacba2-a336-4ca5-83ca-4f9de2ff41a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:08.548838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:08.549955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:08.551608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:08.551608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:08.551608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:08.551608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:08.551608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 478/954] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:08.553351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d025d13-5498-413d-b499-f252e7b136fa None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.556951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabacba2-a336-4ca5-83ca-4f9de2ff41a5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.593667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05dd634-478f-44cb-b756-ad447b10280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.594711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05dd634-478f-44cb-b756-ad447b10280e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.595276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05dd634-478f-44cb-b756-ad447b10280e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.595907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05dd634-478f-44cb-b756-ad447b10280e None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.596831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05dd634-478f-44cb-b756-ad447b10280e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:08.597330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05dd634-478f-44cb-b756-ad447b10280e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.598928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05dd634-478f-44cb-b756-ad447b10280e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.606584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 477/955] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:22:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:08.612309 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05dd634-478f-44cb-b756-ad447b10280e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.671448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabacba2-a336-4ca5-83ca-4f9de2ff41a5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.716051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3ba5b-cc69-4cd5-b8fd-126e82e33a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.717035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3ba5b-cc69-4cd5-b8fd-126e82e33a18 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.717543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3ba5b-cc69-4cd5-b8fd-126e82e33a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.717983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3ba5b-cc69-4cd5-b8fd-126e82e33a18 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.718852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3ba5b-cc69-4cd5-b8fd-126e82e33a18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:08.719317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3ba5b-cc69-4cd5-b8fd-126e82e33a18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.721046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3ba5b-cc69-4cd5-b8fd-126e82e33a18 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.734113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 479/956] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:08.736314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05dd634-478f-44cb-b756-ad447b10280e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.739724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3ba5b-cc69-4cd5-b8fd-126e82e33a18 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.781564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd851-a6f9-4780-9481-24556a28bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.782238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd851-a6f9-4780-9481-24556a28bfc2 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.782519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd851-a6f9-4780-9481-24556a28bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.782784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd851-a6f9-4780-9481-24556a28bfc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.783552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cd851-a6f9-4780-9481-24556a28bfc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.785459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cd851-a6f9-4780-9481-24556a28bfc2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.787138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:08.813882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 478/957] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Jul 2 03:22:08 2022] POST /identity/v3/endpoints => generated 349 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:08.864803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3ba5b-cc69-4cd5-b8fd-126e82e33a18 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:08.917211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550671e-3d5b-4707-959d-669c11202604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:08.918412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550671e-3d5b-4707-959d-669c11202604 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:08.918883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550671e-3d5b-4707-959d-669c11202604 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:08.919380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550671e-3d5b-4707-959d-669c11202604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:08.927770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550671e-3d5b-4707-959d-669c11202604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:08.929602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550671e-3d5b-4707-959d-669c11202604 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b550671e-3d5b-4707-959d-669c11202604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:22:08.946706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:08.947744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:08.948655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:22:08.948655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:22:08.948655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:22:08.948655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:22:08.948655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:22:08.948655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:22:08.948655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:08.948655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:08.951463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 480/958] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:08.958226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd851-a6f9-4780-9481-24556a28bfc2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:08.978824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:09.065721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd851-a6f9-4780-9481-24556a28bfc2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:09.099903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4e882-8c65-4efc-9c3a-f33dba0396ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:09.100912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4e882-8c65-4efc-9c3a-f33dba0396ed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:09.101404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4e882-8c65-4efc-9c3a-f33dba0396ed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:09.101945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4e882-8c65-4efc-9c3a-f33dba0396ed None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:09.103066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4e882-8c65-4efc-9c3a-f33dba0396ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:09.103700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4e882-8c65-4efc-9c3a-f33dba0396ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:09.105385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4e882-8c65-4efc-9c3a-f33dba0396ed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:09.123017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 479/959] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:22:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:09.125053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550671e-3d5b-4707-959d-669c11202604 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:09.224617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:09.231327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550671e-3d5b-4707-959d-669c11202604 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:09.268204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744528c-6f31-417f-a6a7-5d0207d5fdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:09.269219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744528c-6f31-417f-a6a7-5d0207d5fdeb None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:09.269503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744528c-6f31-417f-a6a7-5d0207d5fdeb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:09.269776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744528c-6f31-417f-a6a7-5d0207d5fdeb None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/2fcefa2d73a84415875357ef03e2c8da/roles/d5b0722ac98a4453bc506335e1e71cab` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:09.272037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744528c-6f31-417f-a6a7-5d0207d5fdeb None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=2fcefa2d73a84415875357ef03e2c8da, role_id=d5b0722ac98a4453bc506335e1e71cab)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:09.273477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744528c-6f31-417f-a6a7-5d0207d5fdeb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:09.295495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 481/960] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:22:09 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/2fcefa2d73a84415875357ef03e2c8da/roles/d5b0722ac98a4453bc506335e1e71cab => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:22:09.338738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6f3c5-3de8-4971-912f-671beda6a342 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:09.339454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6f3c5-3de8-4971-912f-671beda6a342 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:09.340030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6f3c5-3de8-4971-912f-671beda6a342 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:09.342032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 480/961] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:09.348018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3f538-a9fd-40b7-9da6-344eca6c29e9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:09.348318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3f538-a9fd-40b7-9da6-344eca6c29e9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:09.348575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3f538-a9fd-40b7-9da6-344eca6c29e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:09.367129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ecd3f538-a9fd-40b7-9da6-344eca6c29e9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:09.368447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ecd3f538-a9fd-40b7-9da6-344eca6c29e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:09.444040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 482/962] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:09.470190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6f3c5-3de8-4971-912f-671beda6a342 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:09.555761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:09.581548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6f3c5-3de8-4971-912f-671beda6a342 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:09.616615 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:09.617305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:09.617653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:09.617959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:09.618683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:09.620212 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:09.636505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.sql.core [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 02 03:22:09.636505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 02 03:22:09.636505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [parameters: {'user_id': '9774ad9c39834ede8f28b2e7cf1373aa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 02 03:22:09.636505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80353) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jul 02 03:22:09.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 02 03:22:09.645208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application yield resource Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.rollback() Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.force_reraise() Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise self.value Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application session.commit() Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 02 03:22:09.646901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._prepare_impl() Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.flush() Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._flush(objects) Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application compat.raise_( Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application flush_context.execute() Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rec.execute(self) Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 02 03:22:09.647856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 02 03:22:09.649955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 02 03:22:09.652088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [parameters: {'user_id': '9774ad9c39834ede8f28b2e7cf1373aa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:09.654326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 02 03:22:09.656387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 02 03:22:09.658485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 02 03:22:09.658485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 02 03:22:09.658485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 02 03:22:09.658485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 02 03:22:09.658485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 02 03:22:09.658485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:09.658485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 481/963] 10.222.0.42 () {60 vars in 1131 bytes} [Sat Jul 2 03:22:09 2022] POST /identity/v3/users => generated 160 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:09.659391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3f538-a9fd-40b7-9da6-344eca6c29e9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:09.748777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:09.766124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3f538-a9fd-40b7-9da6-344eca6c29e9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:09.802117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7e5a4-af95-45b5-8484-19a569582b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:09.803280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7e5a4-af95-45b5-8484-19a569582b04 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:09.803817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7e5a4-af95-45b5-8484-19a569582b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:09.804278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7e5a4-af95-45b5-8484-19a569582b04 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:09.812410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7e5a4-af95-45b5-8484-19a569582b04 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:09.814085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7e5a4-af95-45b5-8484-19a569582b04 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-98b7e5a4-af95-45b5-8484-19a569582b04 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:22:09.828917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:09.830036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 02 03:22:09.830994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:09.833668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 483/964] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:22:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:09.839811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:09.951091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65906e1c-82ea-4c8a-91b0-d322d1f27421 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:09.985556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee064f4-7fb4-4049-8ba1-0cb301d557c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:09.986236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee064f4-7fb4-4049-8ba1-0cb301d557c2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:09.986515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee064f4-7fb4-4049-8ba1-0cb301d557c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:09.986789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee064f4-7fb4-4049-8ba1-0cb301d557c2 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:09.987628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee064f4-7fb4-4049-8ba1-0cb301d557c2 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:09.987934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee064f4-7fb4-4049-8ba1-0cb301d557c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:09.989608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee064f4-7fb4-4049-8ba1-0cb301d557c2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:10.015813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:10.033896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 482/965] 10.222.0.42 () {56 vars in 1101 bytes} [Sat Jul 2 03:22:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:10.596926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effab46-e91e-4aeb-ad8b-0395406e92a8 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:10.597227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effab46-e91e-4aeb-ad8b-0395406e92a8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:10.597614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effab46-e91e-4aeb-ad8b-0395406e92a8 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:10.599190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 484/966] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:10.604530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0344f-f8b4-4faf-bd64-f458a701ca07 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:10.605024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0344f-f8b4-4faf-bd64-f458a701ca07 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:10.605024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0344f-f8b4-4faf-bd64-f458a701ca07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:10.627967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-96a0344f-f8b4-4faf-bd64-f458a701ca07 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:10.629482 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-96a0344f-f8b4-4faf-bd64-f458a701ca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:10.710689 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 483/967] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:10.733097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effab46-e91e-4aeb-ad8b-0395406e92a8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:10.840625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effab46-e91e-4aeb-ad8b-0395406e92a8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:10.876460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915179b-01b0-4cd3-b9ff-99ef740c44af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:10.877045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915179b-01b0-4cd3-b9ff-99ef740c44af None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:10.877326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915179b-01b0-4cd3-b9ff-99ef740c44af None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:10.877599 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915179b-01b0-4cd3-b9ff-99ef740c44af None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:10.878331 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915179b-01b0-4cd3-b9ff-99ef740c44af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:10.879727 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915179b-01b0-4cd3-b9ff-99ef740c44af None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:10.893697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 485/968] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:22:10 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:22:10.927232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de044e31-8728-46f5-90cb-ee2ea24245b8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:10.927598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de044e31-8728-46f5-90cb-ee2ea24245b8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:10.927837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de044e31-8728-46f5-90cb-ee2ea24245b8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:10.929455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 484/969] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:10.935456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019db39-618d-4c98-8bb6-b0a1b909615b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:10.935739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019db39-618d-4c98-8bb6-b0a1b909615b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:10.936054 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019db39-618d-4c98-8bb6-b0a1b909615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:10.948273 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:10.957393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b019db39-618d-4c98-8bb6-b0a1b909615b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:10.959283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b019db39-618d-4c98-8bb6-b0a1b909615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:11.040468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 486/970] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:11.063566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de044e31-8728-46f5-90cb-ee2ea24245b8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:11.182721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de044e31-8728-46f5-90cb-ee2ea24245b8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:11.209095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:11.218439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff18bdb-173d-4f1a-950d-89ce7ae54d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:11.219253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff18bdb-173d-4f1a-950d-89ce7ae54d32 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:11.219566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff18bdb-173d-4f1a-950d-89ce7ae54d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:11.219840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff18bdb-173d-4f1a-950d-89ce7ae54d32 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:11.220499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff18bdb-173d-4f1a-950d-89ce7ae54d32 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:11.221908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff18bdb-173d-4f1a-950d-89ce7ae54d32 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3ff18bdb-173d-4f1a-950d-89ce7ae54d32 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:11.234029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:11.235182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:11.236400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 485/971] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:22:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:11.241519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019db39-618d-4c98-8bb6-b0a1b909615b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:11.344323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019db39-618d-4c98-8bb6-b0a1b909615b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:11.374584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:11.382047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761f0fe-9289-442e-8bb6-22ac5d358252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:11.382718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761f0fe-9289-442e-8bb6-22ac5d358252 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:11.383036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761f0fe-9289-442e-8bb6-22ac5d358252 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:11.383400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761f0fe-9289-442e-8bb6-22ac5d358252 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:11.384088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761f0fe-9289-442e-8bb6-22ac5d358252 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:11.384394 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761f0fe-9289-442e-8bb6-22ac5d358252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:11.385786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761f0fe-9289-442e-8bb6-22ac5d358252 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:11.391071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 487/972] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:22:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:11.396653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff18bdb-173d-4f1a-950d-89ce7ae54d32 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:11.451115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f29d9b-e656-4aed-9b8e-e9d5c09d886d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:11.451342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f29d9b-e656-4aed-9b8e-e9d5c09d886d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:11.451637 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f29d9b-e656-4aed-9b8e-e9d5c09d886d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:11.453123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 488/973] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:11.457791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6118cf1-8921-43da-80c6-7a9392d1ad12 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:11.458083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6118cf1-8921-43da-80c6-7a9392d1ad12 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:11.458381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6118cf1-8921-43da-80c6-7a9392d1ad12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:11.474904 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b6118cf1-8921-43da-80c6-7a9392d1ad12 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:11.476120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b6118cf1-8921-43da-80c6-7a9392d1ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:11.504980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff18bdb-173d-4f1a-950d-89ce7ae54d32 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:11.538654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54079e-8fa2-4c91-b39a-b2087fa7c047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:11.539395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54079e-8fa2-4c91-b39a-b2087fa7c047 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:11.539678 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54079e-8fa2-4c91-b39a-b2087fa7c047 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:11.539962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54079e-8fa2-4c91-b39a-b2087fa7c047 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:11.540665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54079e-8fa2-4c91-b39a-b2087fa7c047 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:11.540974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54079e-8fa2-4c91-b39a-b2087fa7c047 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:11.542356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54079e-8fa2-4c91-b39a-b2087fa7c047 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:11.544733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 489/974] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:11.551344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 486/975] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:22:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:11.553823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6118cf1-8921-43da-80c6-7a9392d1ad12 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:11.571070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54079e-8fa2-4c91-b39a-b2087fa7c047 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:11.631984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:11.657495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6118cf1-8921-43da-80c6-7a9392d1ad12 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:11.679227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54079e-8fa2-4c91-b39a-b2087fa7c047 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:11.682979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:11.692658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6140-7bda-4576-a5a1-4edb6639f879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:11.693317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6140-7bda-4576-a5a1-4edb6639f879 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:11.693603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6140-7bda-4576-a5a1-4edb6639f879 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:11.693880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6140-7bda-4576-a5a1-4edb6639f879 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:11.694675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6140-7bda-4576-a5a1-4edb6639f879 None admin] RBAC: Adding query filter params (interface=admin, service_id=cd34c316a773474097c8922b96188a62, region_id=RegionOne) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:11.694992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6140-7bda-4576-a5a1-4edb6639f879 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:11.696488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6140-7bda-4576-a5a1-4edb6639f879 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:11.701769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 490/976] 10.222.0.42 () {56 vars in 1249 bytes} [Sat Jul 2 03:22:11 2022] GET /identity/v3/endpoints?service_id=cd34c316a773474097c8922b96188a62&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:11.707020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6140-7bda-4576-a5a1-4edb6639f879 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:11.716199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7d084-7a70-4678-aebb-71d522e4b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:11.717018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d084-7a70-4678-aebb-71d522e4b93d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:11.717301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d084-7a70-4678-aebb-71d522e4b93d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:11.717576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7d084-7a70-4678-aebb-71d522e4b93d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:11.728922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7d084-7a70-4678-aebb-71d522e4b93d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-87e7d084-7a70-4678-aebb-71d522e4b93d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:11.737390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:11.738290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:11.739124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 487/977] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Jul 2 03:22:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:11.744348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7d084-7a70-4678-aebb-71d522e4b93d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:11.814910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6140-7bda-4576-a5a1-4edb6639f879 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:11.847424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7d084-7a70-4678-aebb-71d522e4b93d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:11.850951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02733da0-179d-4ece-b111-87121a377774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:11.851649 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02733da0-179d-4ece-b111-87121a377774 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:11.851962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02733da0-179d-4ece-b111-87121a377774 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:11.852243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02733da0-179d-4ece-b111-87121a377774 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:11.852956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02733da0-179d-4ece-b111-87121a377774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:11.854343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02733da0-179d-4ece-b111-87121a377774 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:11.861503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 491/978] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Jul 2 03:22:11 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:22:11.875263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:11.903817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c71f1-7314-4a7b-a15a-ef5695d6d2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:11.904498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c71f1-7314-4a7b-a15a-ef5695d6d2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:11.904781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c71f1-7314-4a7b-a15a-ef5695d6d2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:11.905055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c71f1-7314-4a7b-a15a-ef5695d6d2f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:11.905808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c71f1-7314-4a7b-a15a-ef5695d6d2f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:11.906116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c71f1-7314-4a7b-a15a-ef5695d6d2f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:11.907523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c71f1-7314-4a7b-a15a-ef5695d6d2f3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:11.917374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 488/979] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Jul 2 03:22:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:11.923002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02733da0-179d-4ece-b111-87121a377774 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:11.985789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:12.035703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02733da0-179d-4ece-b111-87121a377774 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:12.070106 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22cfc6-9bc5-4f55-b136-05292de82b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:12.070929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22cfc6-9bc5-4f55-b136-05292de82b12 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:12.071212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22cfc6-9bc5-4f55-b136-05292de82b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:12.071513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22cfc6-9bc5-4f55-b136-05292de82b12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:12.081992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22cfc6-9bc5-4f55-b136-05292de82b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:12.083477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22cfc6-9bc5-4f55-b136-05292de82b12 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-3a22cfc6-9bc5-4f55-b136-05292de82b12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:12.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:12.102781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:12.103854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:12.103854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:12.103854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:12.103854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:12.103854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 492/980] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:12.112158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c71f1-7314-4a7b-a15a-ef5695d6d2f3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:12.214277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c71f1-7314-4a7b-a15a-ef5695d6d2f3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:12.247696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6bcd7-9264-4227-97ea-a959883813d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:12.248274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6bcd7-9264-4227-97ea-a959883813d7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:12.248554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6bcd7-9264-4227-97ea-a959883813d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:12.248824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6bcd7-9264-4227-97ea-a959883813d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:12.249546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6bcd7-9264-4227-97ea-a959883813d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:12.249851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6bcd7-9264-4227-97ea-a959883813d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:12.251216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6bcd7-9264-4227-97ea-a959883813d7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:12.267322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 489/981] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:12.269082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22cfc6-9bc5-4f55-b136-05292de82b12 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:12.371829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22cfc6-9bc5-4f55-b136-05292de82b12 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:12.379268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:12.408445 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a54a65-6900-4f6f-a486-51d4495e7c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:12.409260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a54a65-6900-4f6f-a486-51d4495e7c62 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:12.409541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a54a65-6900-4f6f-a486-51d4495e7c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:12.409814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a54a65-6900-4f6f-a486-51d4495e7c62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:12.416967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a54a65-6900-4f6f-a486-51d4495e7c62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:12.418419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a54a65-6900-4f6f-a486-51d4495e7c62 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-89a54a65-6900-4f6f-a486-51d4495e7c62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:22:12.433695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:12.434733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 02 03:22:12.435729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:12.439596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 493/982] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:22:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:12.440850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6bcd7-9264-4227-97ea-a959883813d7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:12.485881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:12.549346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6bcd7-9264-4227-97ea-a959883813d7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:12.585333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96481d9e-a280-40ee-82b6-206d07c47a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:12.586013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96481d9e-a280-40ee-82b6-206d07c47a16 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:12.586382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96481d9e-a280-40ee-82b6-206d07c47a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:12.586621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96481d9e-a280-40ee-82b6-206d07c47a16 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:12.587524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96481d9e-a280-40ee-82b6-206d07c47a16 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:12.587835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96481d9e-a280-40ee-82b6-206d07c47a16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:12.589276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96481d9e-a280-40ee-82b6-206d07c47a16 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:12.611939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 490/983] 10.222.0.42 () {56 vars in 1149 bytes} [Sat Jul 2 03:22:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:12.613204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a54a65-6900-4f6f-a486-51d4495e7c62 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:12.718907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a54a65-6900-4f6f-a486-51d4495e7c62 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:12.752879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe4c1c-8577-44eb-8d10-1342e0f8e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:12.753525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe4c1c-8577-44eb-8d10-1342e0f8e7ae None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:12.753805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe4c1c-8577-44eb-8d10-1342e0f8e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:12.754080 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe4c1c-8577-44eb-8d10-1342e0f8e7ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:12.787129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:12.822949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe4c1c-8577-44eb-8d10-1342e0f8e7ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:12.824360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe4c1c-8577-44eb-8d10-1342e0f8e7ae None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:12.930732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 494/984] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:12 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 02 03:22:12.932935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96481d9e-a280-40ee-82b6-206d07c47a16 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:12.948057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766db8-f808-4516-b7b2-391033991f2d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:12.948278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766db8-f808-4516-b7b2-391033991f2d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:12.948489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4766db8-f808-4516-b7b2-391033991f2d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:12.949848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 495/985] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:12.955415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea6e78-ef79-4d26-98ad-e99ab91baf67 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:12.955706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea6e78-ef79-4d26-98ad-e99ab91baf67 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:12.955982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea6e78-ef79-4d26-98ad-e99ab91baf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:12.974504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-02ea6e78-ef79-4d26-98ad-e99ab91baf67 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:12.975715 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-02ea6e78-ef79-4d26-98ad-e99ab91baf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:13.040779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:13.044271 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:13.048668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96481d9e-a280-40ee-82b6-206d07c47a16 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.055358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 496/986] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:13.082771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea6e78-ef79-4d26-98ad-e99ab91baf67 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:13.087581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7414fa-e598-4afd-8670-830640e7bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.088587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7414fa-e598-4afd-8670-830640e7bed8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.089023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7414fa-e598-4afd-8670-830640e7bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.089437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7414fa-e598-4afd-8670-830640e7bed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.100207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7414fa-e598-4afd-8670-830640e7bed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.101848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7414fa-e598-4afd-8670-830640e7bed8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0f7414fa-e598-4afd-8670-830640e7bed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:13.123151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:13.124103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:13.130014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:13.130014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:13.130014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:13.130014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:13.130014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 491/987] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:13.130014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7414fa-e598-4afd-8670-830640e7bed8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:13.200136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea6e78-ef79-4d26-98ad-e99ab91baf67 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.238840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185c821-f1f3-4147-935b-44a306065a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.239668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185c821-f1f3-4147-935b-44a306065a18 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.239950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185c821-f1f3-4147-935b-44a306065a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.240396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185c821-f1f3-4147-935b-44a306065a18 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.240959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185c821-f1f3-4147-935b-44a306065a18 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.242244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185c821-f1f3-4147-935b-44a306065a18 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.243388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7414fa-e598-4afd-8670-830640e7bed8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-0185c821-f1f3-4147-935b-44a306065a18 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:13.254626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:13.255665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:13.256491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 497/988] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:13.261842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185c821-f1f3-4147-935b-44a306065a18 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:13.283063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dc723-af30-4189-a819-a37dc40389a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.283823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dc723-af30-4189-a819-a37dc40389a1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.284112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dc723-af30-4189-a819-a37dc40389a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.284417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dc723-af30-4189-a819-a37dc40389a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.285145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dc723-af30-4189-a819-a37dc40389a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:13.285459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dc723-af30-4189-a819-a37dc40389a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.286979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dc723-af30-4189-a819-a37dc40389a1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.304424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 492/989] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:13.306158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dc723-af30-4189-a819-a37dc40389a1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:13.319433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:13.327628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:13.370642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185c821-f1f3-4147-935b-44a306065a18 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.407775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c60624-dea2-4eab-bdcb-021c701cb9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.408559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c60624-dea2-4eab-bdcb-021c701cb9bb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.408893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c60624-dea2-4eab-bdcb-021c701cb9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.409312 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c60624-dea2-4eab-bdcb-021c701cb9bb None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.410032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c60624-dea2-4eab-bdcb-021c701cb9bb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:13.410411 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c60624-dea2-4eab-bdcb-021c701cb9bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.411809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c60624-dea2-4eab-bdcb-021c701cb9bb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.422128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 498/990] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:22:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:13.422884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c60624-dea2-4eab-bdcb-021c701cb9bb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:13.433972 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dc723-af30-4189-a819-a37dc40389a1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.470220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68247308-504e-47cd-a35b-3b0455f2fae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.471016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68247308-504e-47cd-a35b-3b0455f2fae4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.471311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68247308-504e-47cd-a35b-3b0455f2fae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.471604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68247308-504e-47cd-a35b-3b0455f2fae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.478526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68247308-504e-47cd-a35b-3b0455f2fae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.479999 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68247308-504e-47cd-a35b-3b0455f2fae4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-68247308-504e-47cd-a35b-3b0455f2fae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:22:13.494338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:13.495261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:13.496173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:22:13.496173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:22:13.496173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:22:13.496173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:22:13.496173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:22:13.496173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:22:13.496173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:13.496173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:13.496173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 493/991] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:13.501634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68247308-504e-47cd-a35b-3b0455f2fae4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:13.534671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c60624-dea2-4eab-bdcb-021c701cb9bb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.562425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:13.571914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:13.577608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f86be-af65-4e82-bf19-69cc64165bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.578283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f86be-af65-4e82-bf19-69cc64165bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.578563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f86be-af65-4e82-bf19-69cc64165bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.578834 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f86be-af65-4e82-bf19-69cc64165bd9 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.579562 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f86be-af65-4e82-bf19-69cc64165bd9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:13.579865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f86be-af65-4e82-bf19-69cc64165bd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.581517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3f86be-af65-4e82-bf19-69cc64165bd9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.591900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 499/992] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:22:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:13.592924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f86be-af65-4e82-bf19-69cc64165bd9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:13.611202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68247308-504e-47cd-a35b-3b0455f2fae4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.647973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb43bd1-54c8-44b1-b406-dbae803ef4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.648953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb43bd1-54c8-44b1-b406-dbae803ef4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.649230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb43bd1-54c8-44b1-b406-dbae803ef4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.649496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb43bd1-54c8-44b1-b406-dbae803ef4a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.650276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb43bd1-54c8-44b1-b406-dbae803ef4a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:13.650581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb43bd1-54c8-44b1-b406-dbae803ef4a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.652239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb43bd1-54c8-44b1-b406-dbae803ef4a8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.669965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 494/993] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:22:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:13.671079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb43bd1-54c8-44b1-b406-dbae803ef4a8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:13.698139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f86be-af65-4e82-bf19-69cc64165bd9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.733112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c40f8-0cea-4139-bb0d-67e62b50b8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.733770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c40f8-0cea-4139-bb0d-67e62b50b8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.734049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c40f8-0cea-4139-bb0d-67e62b50b8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.734319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c40f8-0cea-4139-bb0d-67e62b50b8d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.735097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c40f8-0cea-4139-bb0d-67e62b50b8d9 None admin] RBAC: Adding query filter params (interface=public, service_id=1f0e62961b4a40ca90d42a3b9dfda0d3, region_id=RegionOne) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:13.735436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c40f8-0cea-4139-bb0d-67e62b50b8d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.736809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663c40f8-0cea-4139-bb0d-67e62b50b8d9 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.742913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 500/994] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Jul 2 03:22:13 2022] GET /identity/v3/endpoints?service_id=1f0e62961b4a40ca90d42a3b9dfda0d3&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:13.748799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c40f8-0cea-4139-bb0d-67e62b50b8d9 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:13.778298 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb43bd1-54c8-44b1-b406-dbae803ef4a8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.805758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:13.806312 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:13.816627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8adc6-9c00-4c16-bc7c-506f16568c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.817286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8adc6-9c00-4c16-bc7c-506f16568c55 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.817566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8adc6-9c00-4c16-bc7c-506f16568c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.817833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8adc6-9c00-4c16-bc7c-506f16568c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.818659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8adc6-9c00-4c16-bc7c-506f16568c55 None admin] RBAC: Adding query filter params (role.id=d5b0722ac98a4453bc506335e1e71cab, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=2fcefa2d73a84415875357ef03e2c8da) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:13.818966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8adc6-9c00-4c16-bc7c-506f16568c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.820528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8adc6-9c00-4c16-bc7c-506f16568c55 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.827701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 495/995] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:13 2022] GET /identity/v3/role_assignments?user.id=2fcefa2d73a84415875357ef03e2c8da&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5b0722ac98a4453bc506335e1e71cab => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:13.863536 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c40f8-0cea-4139-bb0d-67e62b50b8d9 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:13.888832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458ca9d-d19f-4355-a613-1172e1cd24aa None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.889112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458ca9d-d19f-4355-a613-1172e1cd24aa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.889396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458ca9d-d19f-4355-a613-1172e1cd24aa None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.890819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 496/996] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:13.896523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342441a7-85dd-4b1c-9e53-665a155bb6ad None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.896819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342441a7-85dd-4b1c-9e53-665a155bb6ad None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.897087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342441a7-85dd-4b1c-9e53-665a155bb6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.916457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-342441a7-85dd-4b1c-9e53-665a155bb6ad None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:13.917644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-342441a7-85dd-4b1c-9e53-665a155bb6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:13.918028 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a242d-b428-46dd-bf9e-f361bcb33aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:13.918712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a242d-b428-46dd-bf9e-f361bcb33aac None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:13.918978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a242d-b428-46dd-bf9e-f361bcb33aac None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:13.919276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a242d-b428-46dd-bf9e-f361bcb33aac None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:13.919973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a242d-b428-46dd-bf9e-f361bcb33aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:13.921405 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a242d-b428-46dd-bf9e-f361bcb33aac None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:13.928447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 501/997] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Jul 2 03:22:13 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:22:14.009396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 497/998] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:14.032365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a242d-b428-46dd-bf9e-f361bcb33aac None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:14.144277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a242d-b428-46dd-bf9e-f361bcb33aac None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:14.161000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:14.180029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f21d2-8ec9-47dd-9cfd-3814d97408e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:14.180815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f21d2-8ec9-47dd-9cfd-3814d97408e4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:14.181098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f21d2-8ec9-47dd-9cfd-3814d97408e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:14.181371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f21d2-8ec9-47dd-9cfd-3814d97408e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:14.182340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f21d2-8ec9-47dd-9cfd-3814d97408e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:14.183757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f21d2-8ec9-47dd-9cfd-3814d97408e4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-897f21d2-8ec9-47dd-9cfd-3814d97408e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:14.195930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:14.196925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:14.197798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 502/999] 10.222.0.42 () {56 vars in 1126 bytes} [Sat Jul 2 03:22:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:14.205371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342441a7-85dd-4b1c-9e53-665a155bb6ad None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:14.312020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342441a7-85dd-4b1c-9e53-665a155bb6ad None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:14.334841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:14.348522 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75038a2e-aaa0-440f-aabe-2623829232bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:14.349196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75038a2e-aaa0-440f-aabe-2623829232bb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:14.349478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75038a2e-aaa0-440f-aabe-2623829232bb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:14.349752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75038a2e-aaa0-440f-aabe-2623829232bb None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:14.350458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75038a2e-aaa0-440f-aabe-2623829232bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:14.350770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75038a2e-aaa0-440f-aabe-2623829232bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:14.352200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75038a2e-aaa0-440f-aabe-2623829232bb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:14.361756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 498/1000] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:22:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:14.363139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f21d2-8ec9-47dd-9cfd-3814d97408e4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:14.471555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f21d2-8ec9-47dd-9cfd-3814d97408e4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:14.504854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f9e8d-9dec-4104-992f-441b40b31e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:14.505655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f9e8d-9dec-4104-992f-441b40b31e9b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:14.505935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f9e8d-9dec-4104-992f-441b40b31e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:14.506213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f9e8d-9dec-4104-992f-441b40b31e9b None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:14.506911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f9e8d-9dec-4104-992f-441b40b31e9b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:14.507214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f9e8d-9dec-4104-992f-441b40b31e9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:14.508761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f9e8d-9dec-4104-992f-441b40b31e9b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:14.513976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 503/1001] 10.222.0.42 () {56 vars in 1122 bytes} [Sat Jul 2 03:22:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:14.519687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75038a2e-aaa0-440f-aabe-2623829232bb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:14.626007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75038a2e-aaa0-440f-aabe-2623829232bb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:14.661683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd1180-cd70-4d55-9ef6-87d5dca0e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:14.662343 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd1180-cd70-4d55-9ef6-87d5dca0e359 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:14.662634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd1180-cd70-4d55-9ef6-87d5dca0e359 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:14.662876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd1180-cd70-4d55-9ef6-87d5dca0e359 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:14.663732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd1180-cd70-4d55-9ef6-87d5dca0e359 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:14.665217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd1180-cd70-4d55-9ef6-87d5dca0e359 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:14.687337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 499/1002] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Jul 2 03:22:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:15.926675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396029f-6e32-4b75-bd43-bfd1a82f1ac7 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:15.927258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396029f-6e32-4b75-bd43-bfd1a82f1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:15.927258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396029f-6e32-4b75-bd43-bfd1a82f1ac7 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:15.928777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 504/1003] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:15.934302 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb89794-1ae9-4eb0-ae25-e910baf7196d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:15.934742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb89794-1ae9-4eb0-ae25-e910baf7196d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:15.934841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb89794-1ae9-4eb0-ae25-e910baf7196d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:15.952111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7bb89794-1ae9-4eb0-ae25-e910baf7196d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:15.953282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7bb89794-1ae9-4eb0-ae25-e910baf7196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:15.999868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:16.004750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476ef63-856c-40aa-8508-5683ccffbaea None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.005044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476ef63-856c-40aa-8508-5683ccffbaea None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.005310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476ef63-856c-40aa-8508-5683ccffbaea None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.006937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 505/1004] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:16.011837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205709be-057e-4c3e-8e23-afcaffd8b57d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.012125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205709be-057e-4c3e-8e23-afcaffd8b57d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.012398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205709be-057e-4c3e-8e23-afcaffd8b57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.029607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-205709be-057e-4c3e-8e23-afcaffd8b57d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:16.030780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-205709be-057e-4c3e-8e23-afcaffd8b57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:16.034765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 500/1005] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:15 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:16.057530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb89794-1ae9-4eb0-ae25-e910baf7196d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:16.071726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:16.103744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 506/1006] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:16 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:16.123916 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205709be-057e-4c3e-8e23-afcaffd8b57d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:16.176464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb89794-1ae9-4eb0-ae25-e910baf7196d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:16.214330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9c404-7559-4a2d-9adf-e92ea85ee93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:16.215045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9c404-7559-4a2d-9adf-e92ea85ee93b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.215423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9c404-7559-4a2d-9adf-e92ea85ee93b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.215806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9c404-7559-4a2d-9adf-e92ea85ee93b None admin] PATH_INFO: `/v3/services/image` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.216468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9c404-7559-4a2d-9adf-e92ea85ee93b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:16.217852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9c404-7559-4a2d-9adf-e92ea85ee93b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:16.227887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205709be-057e-4c3e-8e23-afcaffd8b57d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-88a9c404-7559-4a2d-9adf-e92ea85ee93b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:16.230211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:16.231261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:16.232108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 501/1007] 10.222.0.42 () {56 vars in 1105 bytes} [Sat Jul 2 03:22:16 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:16.237258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9c404-7559-4a2d-9adf-e92ea85ee93b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:16.267189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b0ee1-4457-4680-9c32-bcf993361028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:16.268018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0ee1-4457-4680-9c32-bcf993361028 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.268299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0ee1-4457-4680-9c32-bcf993361028 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.268573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0ee1-4457-4680-9c32-bcf993361028 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.269228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b0ee1-4457-4680-9c32-bcf993361028 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:16.270602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b0ee1-4457-4680-9c32-bcf993361028 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:16.277194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-899b0ee1-4457-4680-9c32-bcf993361028 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:16.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:16.283414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:16.284253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 507/1008] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:16.289779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b0ee1-4457-4680-9c32-bcf993361028 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:16.333646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:16.351527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9c404-7559-4a2d-9adf-e92ea85ee93b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:16.386449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491bfb7-c09e-4c13-954d-6889908d4100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:16.387120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491bfb7-c09e-4c13-954d-6889908d4100 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.387427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491bfb7-c09e-4c13-954d-6889908d4100 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.387705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491bfb7-c09e-4c13-954d-6889908d4100 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.388533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491bfb7-c09e-4c13-954d-6889908d4100 None admin] RBAC: Adding query filter params (name=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:16.388959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491bfb7-c09e-4c13-954d-6889908d4100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:16.390341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491bfb7-c09e-4c13-954d-6889908d4100 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:16.395730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 502/1009] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:16.401182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b0ee1-4457-4680-9c32-bcf993361028 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:16.401426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491bfb7-c09e-4c13-954d-6889908d4100 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:16.435587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2119f-2c5a-45b6-b5ab-a444f28364fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:16.436260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2119f-2c5a-45b6-b5ab-a444f28364fd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.436542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2119f-2c5a-45b6-b5ab-a444f28364fd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.436819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2119f-2c5a-45b6-b5ab-a444f28364fd None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.437519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2119f-2c5a-45b6-b5ab-a444f28364fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:16.437824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2119f-2c5a-45b6-b5ab-a444f28364fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:16.439276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2119f-2c5a-45b6-b5ab-a444f28364fd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:16.444281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 508/1010] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:22:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:16.450110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2119f-2c5a-45b6-b5ab-a444f28364fd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:16.510665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:16.514025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491bfb7-c09e-4c13-954d-6889908d4100 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:16.549308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ca17f-3e56-4c31-94b1-d298b07f91da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:16.549509 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2119f-2c5a-45b6-b5ab-a444f28364fd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:16.549975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ca17f-3e56-4c31-94b1-d298b07f91da None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.550259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ca17f-3e56-4c31-94b1-d298b07f91da None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.550536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ca17f-3e56-4c31-94b1-d298b07f91da None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.551172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:16.551276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ca17f-3e56-4c31-94b1-d298b07f91da None admin] RBAC: Adding query filter params (type=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:16.551561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ca17f-3e56-4c31-94b1-d298b07f91da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:16.553329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ca17f-3e56-4c31-94b1-d298b07f91da None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:16.558685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 503/1011] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:16.588757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4e260-6cbf-4e6d-b095-09f29b0ac8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:16.589843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4e260-6cbf-4e6d-b095-09f29b0ac8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.590322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4e260-6cbf-4e6d-b095-09f29b0ac8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.590792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4e260-6cbf-4e6d-b095-09f29b0ac8d5 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.591764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4e260-6cbf-4e6d-b095-09f29b0ac8d5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:16.592275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4e260-6cbf-4e6d-b095-09f29b0ac8d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:16.593906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4e260-6cbf-4e6d-b095-09f29b0ac8d5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:16.600353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 509/1012] 10.222.0.42 () {56 vars in 1112 bytes} [Sat Jul 2 03:22:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:16.606039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ca17f-3e56-4c31-94b1-d298b07f91da None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:16.720740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ca17f-3e56-4c31-94b1-d298b07f91da None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:16.755504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6afff-ba5c-45ff-988d-179d281a9e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:16.756177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6afff-ba5c-45ff-988d-179d281a9e68 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.756441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6afff-ba5c-45ff-988d-179d281a9e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.756707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6afff-ba5c-45ff-988d-179d281a9e68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.757447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6afff-ba5c-45ff-988d-179d281a9e68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:16.758970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6afff-ba5c-45ff-988d-179d281a9e68 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:16.762472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b04c5-3ea3-49d9-a56f-891a76666aa2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.762702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b04c5-3ea3-49d9-a56f-891a76666aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.762989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b04c5-3ea3-49d9-a56f-891a76666aa2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.764674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 510/1013] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:16.769748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2537180-53eb-43ca-a9d0-21d0a1b8e958 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:16.770131 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2537180-53eb-43ca-a9d0-21d0a1b8e958 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:16.770416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2537180-53eb-43ca-a9d0-21d0a1b8e958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:16.783114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 504/1014] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Jul 2 03:22:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:22:16.792875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a2537180-53eb-43ca-a9d0-21d0a1b8e958 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:16.794115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a2537180-53eb-43ca-a9d0-21d0a1b8e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:16.881065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 511/1015] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:16 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:16.904554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6afff-ba5c-45ff-988d-179d281a9e68 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:16.930637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:17.019819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6afff-ba5c-45ff-988d-179d281a9e68 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:17.054076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ac98d-f4e7-4cef-966c-9f5d79c5c699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:17.054902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac98d-f4e7-4cef-966c-9f5d79c5c699 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:17.055246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac98d-f4e7-4cef-966c-9f5d79c5c699 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:17.055477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ac98d-f4e7-4cef-966c-9f5d79c5c699 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:17.056975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ac98d-f4e7-4cef-966c-9f5d79c5c699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:17.058418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ac98d-f4e7-4cef-966c-9f5d79c5c699 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:17.060267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 505/1016] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:17.066494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2537180-53eb-43ca-a9d0-21d0a1b8e958 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:17.172047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:17.182267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2537180-53eb-43ca-a9d0-21d0a1b8e958 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:17.217889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d06d4-9932-41d8-a6f4-a9b557404521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:17.218557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d06d4-9932-41d8-a6f4-a9b557404521 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:17.218841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d06d4-9932-41d8-a6f4-a9b557404521 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:17.219118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d06d4-9932-41d8-a6f4-a9b557404521 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:17.219876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d06d4-9932-41d8-a6f4-a9b557404521 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:17.221320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d06d4-9932-41d8-a6f4-a9b557404521 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:17.248443 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 512/1017] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:22:17 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:22:18.658786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe27f28-237a-43c6-9867-49670b837f21 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:18.659365 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe27f28-237a-43c6-9867-49670b837f21 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:18.659365 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe27f28-237a-43c6-9867-49670b837f21 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:18.660929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 506/1018] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:18.666479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd645e-ca85-4838-af05-ee28805ddc46 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:18.666903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd645e-ca85-4838-af05-ee28805ddc46 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:18.667077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd645e-ca85-4838-af05-ee28805ddc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:18.685061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-11cd645e-ca85-4838-af05-ee28805ddc46 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:18.686255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-11cd645e-ca85-4838-af05-ee28805ddc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:18.757457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 513/1019] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:18 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:18.780728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe27f28-237a-43c6-9867-49670b837f21 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:18.876280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:18.891036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe27f28-237a-43c6-9867-49670b837f21 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:18.917043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b4ceb-c113-47e2-a75f-a8180337f911 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:18.917212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b4ceb-c113-47e2-a75f-a8180337f911 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:18.917485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b4ceb-c113-47e2-a75f-a8180337f911 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:18.919051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 514/1020] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:18.924007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4726f-56bc-4d07-8829-740e8eb155fa None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:18.924360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4726f-56bc-4d07-8829-740e8eb155fa None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:18.924702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4726f-56bc-4d07-8829-740e8eb155fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:18.925526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dbbdf-9829-43d5-844d-03a3e0f52bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:18.926183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dbbdf-9829-43d5-844d-03a3e0f52bea None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:18.926472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dbbdf-9829-43d5-844d-03a3e0f52bea None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:18.926739 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dbbdf-9829-43d5-844d-03a3e0f52bea None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:18.927494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dbbdf-9829-43d5-844d-03a3e0f52bea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:18.928867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dbbdf-9829-43d5-844d-03a3e0f52bea None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:18.942384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e0a4726f-56bc-4d07-8829-740e8eb155fa None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:18.943683 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e0a4726f-56bc-4d07-8829-740e8eb155fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:18.945883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 507/1021] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:22:18 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:19.019316 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:19.028969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 515/1022] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:19.052213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dbbdf-9829-43d5-844d-03a3e0f52bea None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:19.165187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dbbdf-9829-43d5-844d-03a3e0f52bea None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:19.198772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449eef75-275f-4a12-92ef-48a3789f4e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:19.199526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eef75-275f-4a12-92ef-48a3789f4e91 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:19.199805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eef75-275f-4a12-92ef-48a3789f4e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:19.200074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eef75-275f-4a12-92ef-48a3789f4e91 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:19.210319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449eef75-275f-4a12-92ef-48a3789f4e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-449eef75-275f-4a12-92ef-48a3789f4e91 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:19.218539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:19.219753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:19.220856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 508/1023] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Jul 2 03:22:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:19.226007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4726f-56bc-4d07-8829-740e8eb155fa None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:19.328179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4726f-56bc-4d07-8829-740e8eb155fa None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:19.365819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb6a59-7a12-4633-a73b-597cf9e60e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:19.366487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb6a59-7a12-4633-a73b-597cf9e60e7f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:19.366766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb6a59-7a12-4633-a73b-597cf9e60e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:19.367039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb6a59-7a12-4633-a73b-597cf9e60e7f None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:19.367905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb6a59-7a12-4633-a73b-597cf9e60e7f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:19.368207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb6a59-7a12-4633-a73b-597cf9e60e7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:19.369650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb6a59-7a12-4633-a73b-597cf9e60e7f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:19.377976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 516/1024] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Jul 2 03:22:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:19.379174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a64af-5cd9-4c2e-812c-442f699cf332 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:19.379569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a64af-5cd9-4c2e-812c-442f699cf332 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:19.379913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a64af-5cd9-4c2e-812c-442f699cf332 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:19.381474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 509/1025] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:19.383396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb6a59-7a12-4633-a73b-597cf9e60e7f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:19.387367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63c4fa-e68b-4b45-9d1b-cd196945bd5a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:19.387655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63c4fa-e68b-4b45-9d1b-cd196945bd5a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:19.387930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63c4fa-e68b-4b45-9d1b-cd196945bd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:19.404670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1c63c4fa-e68b-4b45-9d1b-cd196945bd5a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:19.405856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1c63c4fa-e68b-4b45-9d1b-cd196945bd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:19.407872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:19.451797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:19.487404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 510/1026] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:19.492003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb6a59-7a12-4633-a73b-597cf9e60e7f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:19.508255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63c4fa-e68b-4b45-9d1b-cd196945bd5a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:19.526473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3aa47-0cf8-481b-adc1-029783f416db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:19.527267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3aa47-0cf8-481b-adc1-029783f416db None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:19.527577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3aa47-0cf8-481b-adc1-029783f416db None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:19.527849 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3aa47-0cf8-481b-adc1-029783f416db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:19.538324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3aa47-0cf8-481b-adc1-029783f416db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:19.540137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3aa47-0cf8-481b-adc1-029783f416db None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7aa3aa47-0cf8-481b-adc1-029783f416db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:19.557878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:19.558940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:19.560045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:19.560045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:19.560045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:19.560045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:19.560045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 517/1027] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:19.564969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3aa47-0cf8-481b-adc1-029783f416db None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:19.612631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63c4fa-e68b-4b45-9d1b-cd196945bd5a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:19.653559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66205602-ce0d-45d4-bc06-bb4cd0cf4bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:19.654661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66205602-ce0d-45d4-bc06-bb4cd0cf4bbd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:19.655051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66205602-ce0d-45d4-bc06-bb4cd0cf4bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:19.655498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66205602-ce0d-45d4-bc06-bb4cd0cf4bbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:19.657235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66205602-ce0d-45d4-bc06-bb4cd0cf4bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:19.659540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66205602-ce0d-45d4-bc06-bb4cd0cf4bbd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:19.667203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 511/1028] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:19.668143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66205602-ce0d-45d4-bc06-bb4cd0cf4bbd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:19.671442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3aa47-0cf8-481b-adc1-029783f416db None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:19.678391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:19.689887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:19.713018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a2e71-e371-4aa8-a9a5-672ce1ce64da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:19.713607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a2e71-e371-4aa8-a9a5-672ce1ce64da None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:19.713895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a2e71-e371-4aa8-a9a5-672ce1ce64da None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:19.714176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a2e71-e371-4aa8-a9a5-672ce1ce64da None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:19.714906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a2e71-e371-4aa8-a9a5-672ce1ce64da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:19.715216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a2e71-e371-4aa8-a9a5-672ce1ce64da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:19.716642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a2e71-e371-4aa8-a9a5-672ce1ce64da None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:19.729244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 518/1029] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:19.734772 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a2e71-e371-4aa8-a9a5-672ce1ce64da None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:19.781303 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66205602-ce0d-45d4-bc06-bb4cd0cf4bbd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:19.817198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c75239-6b60-4f71-b366-56b53450cc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:19.817864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c75239-6b60-4f71-b366-56b53450cc9f None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:19.818147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c75239-6b60-4f71-b366-56b53450cc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:19.818415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c75239-6b60-4f71-b366-56b53450cc9f None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:19.819113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c75239-6b60-4f71-b366-56b53450cc9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:19.820729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c75239-6b60-4f71-b366-56b53450cc9f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:19.841121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 512/1030] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:22:19 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:19.845946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a2e71-e371-4aa8-a9a5-672ce1ce64da None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:19.884703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8583d5-8d1a-42a0-9904-4e4623a8eece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:19.885513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8583d5-8d1a-42a0-9904-4e4623a8eece None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:19.885799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8583d5-8d1a-42a0-9904-4e4623a8eece None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:19.886076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8583d5-8d1a-42a0-9904-4e4623a8eece None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:19.894490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8583d5-8d1a-42a0-9904-4e4623a8eece None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:19.896015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8583d5-8d1a-42a0-9904-4e4623a8eece None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-7b8583d5-8d1a-42a0-9904-4e4623a8eece None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:22:19.911954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:19.912996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:19.919392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 519/1031] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Jul 2 03:22:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:19.921955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c75239-6b60-4f71-b366-56b53450cc9f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:19.991027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:20.040713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c75239-6b60-4f71-b366-56b53450cc9f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:20.081805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c9153-950b-4d26-9d1f-983203a106e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:20.082492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c9153-950b-4d26-9d1f-983203a106e1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:20.082770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c9153-950b-4d26-9d1f-983203a106e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:20.083041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c9153-950b-4d26-9d1f-983203a106e1 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:20.083933 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c9153-950b-4d26-9d1f-983203a106e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:20.084246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c9153-950b-4d26-9d1f-983203a106e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:20.085738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c9153-950b-4d26-9d1f-983203a106e1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:20.105250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 513/1032] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Jul 2 03:22:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:20.111199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8583d5-8d1a-42a0-9904-4e4623a8eece None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:20.148470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:20.218002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8583d5-8d1a-42a0-9904-4e4623a8eece None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:20.254667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823c8a4-fd15-489c-907a-af2eaa60ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:20.255230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823c8a4-fd15-489c-907a-af2eaa60ce9c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:20.255543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823c8a4-fd15-489c-907a-af2eaa60ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:20.255824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823c8a4-fd15-489c-907a-af2eaa60ce9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:20.323782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823c8a4-fd15-489c-907a-af2eaa60ce9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:20.325248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823c8a4-fd15-489c-907a-af2eaa60ce9c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:20.362085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:20.437398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 520/1033] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:20 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 02 03:22:20.442997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c9153-950b-4d26-9d1f-983203a106e1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:20.543321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c9153-950b-4d26-9d1f-983203a106e1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:20.563659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:20.579329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e9a2f-3443-410a-abc3-b21e4c319ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:20.580147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e9a2f-3443-410a-abc3-b21e4c319ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:20.580431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e9a2f-3443-410a-abc3-b21e4c319ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:20.580703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e9a2f-3443-410a-abc3-b21e4c319ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:20.591247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e9a2f-3443-410a-abc3-b21e4c319ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:20.592738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e9a2f-3443-410a-abc3-b21e4c319ca3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e10e9a2f-3443-410a-abc3-b21e4c319ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:20.610994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:20.612118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:20.613015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:20.613015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:20.613015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:20.613015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:20.613871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 514/1034] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:20.617933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823c8a4-fd15-489c-907a-af2eaa60ce9c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:20.725851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823c8a4-fd15-489c-907a-af2eaa60ce9c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:20.760804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f57bc-0116-409b-91f7-5ae746547a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:20.761466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f57bc-0116-409b-91f7-5ae746547a52 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:20.761745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f57bc-0116-409b-91f7-5ae746547a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:20.762015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f57bc-0116-409b-91f7-5ae746547a52 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:20.762731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f57bc-0116-409b-91f7-5ae746547a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:20.763035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f57bc-0116-409b-91f7-5ae746547a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:20.764442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f57bc-0116-409b-91f7-5ae746547a52 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:20.765862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:20.782131 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 521/1035] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:20.783487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e9a2f-3443-410a-abc3-b21e4c319ca3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:20.886723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e9a2f-3443-410a-abc3-b21e4c319ca3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:20.920421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f4915-e77a-4804-85cd-c33013a47ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:20.921222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f4915-e77a-4804-85cd-c33013a47ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:20.921498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f4915-e77a-4804-85cd-c33013a47ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:20.921768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f4915-e77a-4804-85cd-c33013a47ae0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:20.928554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f4915-e77a-4804-85cd-c33013a47ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:20.930201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f4915-e77a-4804-85cd-c33013a47ae0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-403f4915-e77a-4804-85cd-c33013a47ae0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:22:20.944588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:20.945540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:20.946445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:22:20.946445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:22:20.946445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:22:20.946445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:22:20.946445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:22:20.946445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:22:20.946445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:20.946445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:20.950597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 515/1036] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:20.951660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f57bc-0116-409b-91f7-5ae746547a52 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:21.012171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94e634-a717-46e8-8258-343524d5ceab None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.012506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94e634-a717-46e8-8258-343524d5ceab None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.012720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94e634-a717-46e8-8258-343524d5ceab None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.014250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 516/1037] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:21.018848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9be7-fe32-4dac-b7a3-d597376f42ea None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.019117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9be7-fe32-4dac-b7a3-d597376f42ea None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.019422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a9be7-fe32-4dac-b7a3-d597376f42ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.036053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-443a9be7-fe32-4dac-b7a3-d597376f42ea None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:21.037228 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-443a9be7-fe32-4dac-b7a3-d597376f42ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:21.059147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f57bc-0116-409b-91f7-5ae746547a52 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:21.065748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:21.095675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4624892-2d68-4710-9c00-123b95408dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:21.096335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4624892-2d68-4710-9c00-123b95408dee None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.096617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4624892-2d68-4710-9c00-123b95408dee None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.096884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4624892-2d68-4710-9c00-123b95408dee None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.097666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4624892-2d68-4710-9c00-123b95408dee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:21.097971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4624892-2d68-4710-9c00-123b95408dee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:21.099438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4624892-2d68-4710-9c00-123b95408dee None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:21.112322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 517/1038] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:21.114496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 522/1039] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:22:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:21.120322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9be7-fe32-4dac-b7a3-d597376f42ea None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:21.134953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4624892-2d68-4710-9c00-123b95408dee None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:21.192506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:21.225982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a9be7-fe32-4dac-b7a3-d597376f42ea None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:21.241352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4624892-2d68-4710-9c00-123b95408dee None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:21.259489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d56b-a24e-4a73-80ef-7ec4c789168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:21.260146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d56b-a24e-4a73-80ef-7ec4c789168c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.260427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d56b-a24e-4a73-80ef-7ec4c789168c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.260700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d56b-a24e-4a73-80ef-7ec4c789168c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.261533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423d56b-a24e-4a73-80ef-7ec4c789168c None admin] RBAC: Adding query filter params (role.id=d5b0722ac98a4453bc506335e1e71cab, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=8b7bc20034974816905978dd365993f8) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:21.261843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423d56b-a24e-4a73-80ef-7ec4c789168c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:21.263325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423d56b-a24e-4a73-80ef-7ec4c789168c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:21.269418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 518/1040] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:21 2022] GET /identity/v3/role_assignments?user.id=8b7bc20034974816905978dd365993f8&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5b0722ac98a4453bc506335e1e71cab => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:21.280357 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0b33d-01b8-4e98-a723-ccbd7cc74d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:21.281174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0b33d-01b8-4e98-a723-ccbd7cc74d99 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.281450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0b33d-01b8-4e98-a723-ccbd7cc74d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.281717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0b33d-01b8-4e98-a723-ccbd7cc74d99 None admin] PATH_INFO: `/v3/services/image` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.282374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0b33d-01b8-4e98-a723-ccbd7cc74d99 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:21.283827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0b33d-01b8-4e98-a723-ccbd7cc74d99 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-8ea0b33d-01b8-4e98-a723-ccbd7cc74d99 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:21.296924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:21.297998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 02 03:22:21.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:21.300632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 523/1041] 10.222.0.42 () {56 vars in 1105 bytes} [Sat Jul 2 03:22:21 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:21.304430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d56b-a24e-4a73-80ef-7ec4c789168c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:21.335788 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:21.411456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d56b-a24e-4a73-80ef-7ec4c789168c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:21.451140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418875b-4769-433d-9ad1-75db9a7ce59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:21.451760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418875b-4769-433d-9ad1-75db9a7ce59d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.452040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418875b-4769-433d-9ad1-75db9a7ce59d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.452311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a418875b-4769-433d-9ad1-75db9a7ce59d None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.453009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418875b-4769-433d-9ad1-75db9a7ce59d None admin] RBAC: Adding query filter params (name=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:21.453311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418875b-4769-433d-9ad1-75db9a7ce59d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:21.454682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a418875b-4769-433d-9ad1-75db9a7ce59d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:21.460818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 519/1042] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:21.465911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0b33d-01b8-4e98-a723-ccbd7cc74d99 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:21.567953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0b33d-01b8-4e98-a723-ccbd7cc74d99 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:21.585479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:21.604649 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ad6c9-62e0-4608-95f4-a7f58592c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:21.605315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ad6c9-62e0-4608-95f4-a7f58592c01c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.605596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ad6c9-62e0-4608-95f4-a7f58592c01c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.605868 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ad6c9-62e0-4608-95f4-a7f58592c01c None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.606568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ad6c9-62e0-4608-95f4-a7f58592c01c None admin] RBAC: Adding query filter params (type=image) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:21.606875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ad6c9-62e0-4608-95f4-a7f58592c01c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:21.608288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ad6c9-62e0-4608-95f4-a7f58592c01c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:21.618324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 524/1043] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:21.619996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418875b-4769-433d-9ad1-75db9a7ce59d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:21.716024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:21.727253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a418875b-4769-433d-9ad1-75db9a7ce59d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:21.761590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac6034-b4ca-4b17-8126-93177965ff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:21.762245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac6034-b4ca-4b17-8126-93177965ff91 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.762524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac6034-b4ca-4b17-8126-93177965ff91 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.762797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac6034-b4ca-4b17-8126-93177965ff91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.763610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac6034-b4ca-4b17-8126-93177965ff91 None admin] RBAC: Adding query filter params (interface=public, service_id=97a2da239d88476ea7ce548840603abd, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:21.763919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac6034-b4ca-4b17-8126-93177965ff91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:21.765289 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac6034-b4ca-4b17-8126-93177965ff91 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:21.770766 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 520/1044] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Jul 2 03:22:21 2022] GET /identity/v3/endpoints?service_id=97a2da239d88476ea7ce548840603abd&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:21.776650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ad6c9-62e0-4608-95f4-a7f58592c01c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:21.876559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ad6c9-62e0-4608-95f4-a7f58592c01c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:21.910911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76faf20-f105-434c-8711-aedb53348341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:21.911591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76faf20-f105-434c-8711-aedb53348341 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.911877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76faf20-f105-434c-8711-aedb53348341 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.912143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76faf20-f105-434c-8711-aedb53348341 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.912813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76faf20-f105-434c-8711-aedb53348341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:21.914222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76faf20-f105-434c-8711-aedb53348341 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:21.921123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 525/1045] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Jul 2 03:22:21 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:22:21.923929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58fade-f212-4164-ad3c-4324e1591381 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.924449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58fade-f212-4164-ad3c-4324e1591381 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.924449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f58fade-f212-4164-ad3c-4324e1591381 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.926076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 521/1046] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:21.932088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233e03c-5edc-4261-9e9c-50c8311bdbd6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:21.932453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233e03c-5edc-4261-9e9c-50c8311bdbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:21.932645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233e03c-5edc-4261-9e9c-50c8311bdbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:21.951230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9233e03c-5edc-4261-9e9c-50c8311bdbd6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:21.952488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9233e03c-5edc-4261-9e9c-50c8311bdbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:22.002185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:22.035207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 526/1047] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:22.062101 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58fade-f212-4164-ad3c-4324e1591381 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:22.177494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f58fade-f212-4164-ad3c-4324e1591381 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:22.213775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8642-6cc0-46de-ab86-9890ddaeb042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:22.214564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8642-6cc0-46de-ab86-9890ddaeb042 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:22.214847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8642-6cc0-46de-ab86-9890ddaeb042 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:22.215071 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8642-6cc0-46de-ab86-9890ddaeb042 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:22.225470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d8642-6cc0-46de-ab86-9890ddaeb042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-bc1d8642-6cc0-46de-ab86-9890ddaeb042 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:22.233036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:22.234003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:22.234865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 522/1048] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:22:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:22.240199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233e03c-5edc-4261-9e9c-50c8311bdbd6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:22.352487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233e03c-5edc-4261-9e9c-50c8311bdbd6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:22.390625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd30e99-6224-4e22-ab75-f144ac8d2a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:22.391324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd30e99-6224-4e22-ab75-f144ac8d2a7c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:22.391613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd30e99-6224-4e22-ab75-f144ac8d2a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:22.391891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd30e99-6224-4e22-ab75-f144ac8d2a7c None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:22.392642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd30e99-6224-4e22-ab75-f144ac8d2a7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:22.392954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd30e99-6224-4e22-ab75-f144ac8d2a7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:22.394343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd30e99-6224-4e22-ab75-f144ac8d2a7c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:22.403734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 527/1049] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:22:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:22.408866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8642-6cc0-46de-ab86-9890ddaeb042 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:22.425994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:22.514512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8642-6cc0-46de-ab86-9890ddaeb042 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:22.548154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fca0d-9b26-4f76-ac08-b704f2bfdb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:22.549049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fca0d-9b26-4f76-ac08-b704f2bfdb98 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:22.549334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fca0d-9b26-4f76-ac08-b704f2bfdb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:22.549609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fca0d-9b26-4f76-ac08-b704f2bfdb98 None admin] PATH_INFO: `/v3/users/6e96456378b14f33a85ec4804fe8488e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:22.563501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fca0d-9b26-4f76-ac08-b704f2bfdb98 None admin] RBAC: Authorizing `identity:get_user(user_id=6e96456378b14f33a85ec4804fe8488e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:22.564960 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fca0d-9b26-4f76-ac08-b704f2bfdb98 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:22.571329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 523/1050] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:22 2022] GET /identity/v3/users/6e96456378b14f33a85ec4804fe8488e => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:22.572875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd30e99-6224-4e22-ab75-f144ac8d2a7c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:22.614960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:22.676407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd30e99-6224-4e22-ab75-f144ac8d2a7c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:22.712919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b92cc8-5bcc-43c4-a9f1-5ff02bb8f079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:22.713567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b92cc8-5bcc-43c4-a9f1-5ff02bb8f079 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:22.713845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b92cc8-5bcc-43c4-a9f1-5ff02bb8f079 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:22.714113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b92cc8-5bcc-43c4-a9f1-5ff02bb8f079 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:22.781177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b92cc8-5bcc-43c4-a9f1-5ff02bb8f079 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:22.782612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b92cc8-5bcc-43c4-a9f1-5ff02bb8f079 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:22.821075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:22.881895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 528/1051] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:22 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:22:22.887862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fca0d-9b26-4f76-ac08-b704f2bfdb98 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:22.959647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:22.993192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fca0d-9b26-4f76-ac08-b704f2bfdb98 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:23.028758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd250f-f01f-4f90-8dfe-1c5aa2644d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:23.029622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd250f-f01f-4f90-8dfe-1c5aa2644d56 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:23.029900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd250f-f01f-4f90-8dfe-1c5aa2644d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:23.030173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd250f-f01f-4f90-8dfe-1c5aa2644d56 None admin] PATH_INFO: `/v3/projects/0fd344da8a9648ec8c2e2b28593e8708` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:23.031378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd250f-f01f-4f90-8dfe-1c5aa2644d56 None admin] RBAC: Authorizing `identity:get_project(project_id=0fd344da8a9648ec8c2e2b28593e8708)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:23.032810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd250f-f01f-4f90-8dfe-1c5aa2644d56 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:23.034698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 524/1052] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:22:22 2022] GET /identity/v3/projects/0fd344da8a9648ec8c2e2b28593e8708 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:23.041232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b92cc8-5bcc-43c4-a9f1-5ff02bb8f079 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:23.151427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b92cc8-5bcc-43c4-a9f1-5ff02bb8f079 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:23.186098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479fee1-0a3b-439f-b170-3950cc808768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:23.186748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479fee1-0a3b-439f-b170-3950cc808768 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:23.187026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479fee1-0a3b-439f-b170-3950cc808768 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:23.187337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479fee1-0a3b-439f-b170-3950cc808768 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:23.188141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479fee1-0a3b-439f-b170-3950cc808768 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=0fd344da8a9648ec8c2e2b28593e8708, user.id=6e96456378b14f33a85ec4804fe8488e) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:23.188451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479fee1-0a3b-439f-b170-3950cc808768 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:23.189886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479fee1-0a3b-439f-b170-3950cc808768 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:23.195725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 529/1053] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:23 2022] GET /identity/v3/role_assignments?user.id=6e96456378b14f33a85ec4804fe8488e&scope.project.id=0fd344da8a9648ec8c2e2b28593e8708&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:23.280651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d56447-49b1-4e8f-bad3-5898d8252cae None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:23.281006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d56447-49b1-4e8f-bad3-5898d8252cae None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:23.281166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d56447-49b1-4e8f-bad3-5898d8252cae None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:23.282779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 525/1054] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:23.288346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab442402-1c72-41f1-b851-aac806518ea7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:23.288638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab442402-1c72-41f1-b851-aac806518ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:23.288902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab442402-1c72-41f1-b851-aac806518ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:23.294603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:23.310003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ab442402-1c72-41f1-b851-aac806518ea7 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:23.311238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ab442402-1c72-41f1-b851-aac806518ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:23.385737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 530/1055] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:23.407961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d56447-49b1-4e8f-bad3-5898d8252cae None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:23.511749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d56447-49b1-4e8f-bad3-5898d8252cae None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:23.546660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72948f-c650-426a-a0ec-5669bd02dfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:23.547495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72948f-c650-426a-a0ec-5669bd02dfcb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:23.547774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72948f-c650-426a-a0ec-5669bd02dfcb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:23.548045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72948f-c650-426a-a0ec-5669bd02dfcb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:23.556004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:23.559972 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72948f-c650-426a-a0ec-5669bd02dfcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0c72948f-c650-426a-a0ec-5669bd02dfcb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:23.568601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:23.569578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:23.570511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:23.571947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 526/1056] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Jul 2 03:22:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:23.576094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab442402-1c72-41f1-b851-aac806518ea7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:23.677162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab442402-1c72-41f1-b851-aac806518ea7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:23.707326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:23.714003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8a945-063d-46c8-92a2-3d332a380fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:23.714668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8a945-063d-46c8-92a2-3d332a380fcd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:23.714942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8a945-063d-46c8-92a2-3d332a380fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:23.715215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8a945-063d-46c8-92a2-3d332a380fcd None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:23.715983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8a945-063d-46c8-92a2-3d332a380fcd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:23.716285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8a945-063d-46c8-92a2-3d332a380fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:23.717660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8a945-063d-46c8-92a2-3d332a380fcd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:23.730502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 531/1057] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Jul 2 03:22:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:23.731877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72948f-c650-426a-a0ec-5669bd02dfcb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:23.831458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72948f-c650-426a-a0ec-5669bd02dfcb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:23.865630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cf2e2-1276-4e34-a3bf-b017343fb4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:23.866430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cf2e2-1276-4e34-a3bf-b017343fb4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:23.866726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cf2e2-1276-4e34-a3bf-b017343fb4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:23.866995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cf2e2-1276-4e34-a3bf-b017343fb4f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:23.878316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cf2e2-1276-4e34-a3bf-b017343fb4f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:23.879797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cf2e2-1276-4e34-a3bf-b017343fb4f2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-089cf2e2-1276-4e34-a3bf-b017343fb4f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:23.897197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:23.898139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:23.899165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:23.899165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:23.899165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:23.899165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:23.903555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 527/1058] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:23.906103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8a945-063d-46c8-92a2-3d332a380fcd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:23.965556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4039d-9d6f-4c71-a563-ddd5ea6fd9be None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:23.966066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4039d-9d6f-4c71-a563-ddd5ea6fd9be None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:23.966066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4039d-9d6f-4c71-a563-ddd5ea6fd9be None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:23.967788 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 528/1059] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:23.972574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400be1d-bfd9-4af8-a7b1-2dd27d9b2b6b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:23.972859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400be1d-bfd9-4af8-a7b1-2dd27d9b2b6b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:23.973135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400be1d-bfd9-4af8-a7b1-2dd27d9b2b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:23.990618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7400be1d-bfd9-4af8-a7b1-2dd27d9b2b6b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:23.991840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7400be1d-bfd9-4af8-a7b1-2dd27d9b2b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:24.020197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8a945-063d-46c8-92a2-3d332a380fcd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.040911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:24.062193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 529/1060] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:24.064153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894269d-6aab-43a4-823f-292c869cf39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.065006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894269d-6aab-43a4-823f-292c869cf39b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.065457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894269d-6aab-43a4-823f-292c869cf39b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.065877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894269d-6aab-43a4-823f-292c869cf39b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.066736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894269d-6aab-43a4-823f-292c869cf39b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:24.067188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894269d-6aab-43a4-823f-292c869cf39b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.068713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894269d-6aab-43a4-823f-292c869cf39b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.085240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400be1d-bfd9-4af8-a7b1-2dd27d9b2b6b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:24.085433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 532/1061] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:24.086519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894269d-6aab-43a4-823f-292c869cf39b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:24.150125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:24.191806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400be1d-bfd9-4af8-a7b1-2dd27d9b2b6b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.201772 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894269d-6aab-43a4-823f-292c869cf39b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.229621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8db108-28f2-4629-be57-1dbbca5ea8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.230418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8db108-28f2-4629-be57-1dbbca5ea8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.230702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8db108-28f2-4629-be57-1dbbca5ea8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.230977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8db108-28f2-4629-be57-1dbbca5ea8b3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.231665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8db108-28f2-4629-be57-1dbbca5ea8b3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.233148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8db108-28f2-4629-be57-1dbbca5ea8b3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.237826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f90dd-e4f9-45ca-8338-d969bdcaa223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.238618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f90dd-e4f9-45ca-8338-d969bdcaa223 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.238901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f90dd-e4f9-45ca-8338-d969bdcaa223 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.239177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f90dd-e4f9-45ca-8338-d969bdcaa223 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2e8db108-28f2-4629-be57-1dbbca5ea8b3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:24.244969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:24.245945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:24.246761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 530/1062] 10.222.0.42 () {56 vars in 1105 bytes} [Sat Jul 2 03:22:24 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:24.249033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f90dd-e4f9-45ca-8338-d969bdcaa223 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.250497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f90dd-e4f9-45ca-8338-d969bdcaa223 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.252083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8db108-28f2-4629-be57-1dbbca5ea8b3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-f48f90dd-e4f9-45ca-8338-d969bdcaa223 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:22:24.265626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:24.266655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:24.268134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 533/1063] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Jul 2 03:22:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:24.272852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f90dd-e4f9-45ca-8338-d969bdcaa223 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:24.304827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:24.362388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8db108-28f2-4629-be57-1dbbca5ea8b3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.378206 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f90dd-e4f9-45ca-8338-d969bdcaa223 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.393002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:24.398526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e42be3-09f3-4b33-9ee1-06dd1e12afff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.399437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42be3-09f3-4b33-9ee1-06dd1e12afff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.399866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42be3-09f3-4b33-9ee1-06dd1e12afff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.400284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e42be3-09f3-4b33-9ee1-06dd1e12afff None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.401153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e42be3-09f3-4b33-9ee1-06dd1e12afff None admin] RBAC: Adding query filter params (name=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:24.401618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e42be3-09f3-4b33-9ee1-06dd1e12afff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.403007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e42be3-09f3-4b33-9ee1-06dd1e12afff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.408423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 531/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:24.413719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e42be3-09f3-4b33-9ee1-06dd1e12afff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:24.418881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6ea49-f51f-4c28-aae0-f4506ce25f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.419577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ea49-f51f-4c28-aae0-f4506ce25f57 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.419856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ea49-f51f-4c28-aae0-f4506ce25f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.420126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ea49-f51f-4c28-aae0-f4506ce25f57 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.420959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6ea49-f51f-4c28-aae0-f4506ce25f57 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:24.421268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6ea49-f51f-4c28-aae0-f4506ce25f57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.422654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6ea49-f51f-4c28-aae0-f4506ce25f57 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.440632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 534/1065] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Jul 2 03:22:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:24.446339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6ea49-f51f-4c28-aae0-f4506ce25f57 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:24.514931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e42be3-09f3-4b33-9ee1-06dd1e12afff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.526823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:24.551257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a311b75-5aab-4a7d-9d5b-6dcee1d34c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.551757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6ea49-f51f-4c28-aae0-f4506ce25f57 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.551957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a311b75-5aab-4a7d-9d5b-6dcee1d34c4d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.552247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a311b75-5aab-4a7d-9d5b-6dcee1d34c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.552527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a311b75-5aab-4a7d-9d5b-6dcee1d34c4d None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.553394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a311b75-5aab-4a7d-9d5b-6dcee1d34c4d None admin] RBAC: Adding query filter params (type=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:24.553710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a311b75-5aab-4a7d-9d5b-6dcee1d34c4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.556052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a311b75-5aab-4a7d-9d5b-6dcee1d34c4d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.566237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 532/1066] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:24.567623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a311b75-5aab-4a7d-9d5b-6dcee1d34c4d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:24.587618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f0553-73d8-4814-85b8-a32e264b89f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.588420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f0553-73d8-4814-85b8-a32e264b89f4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.588708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f0553-73d8-4814-85b8-a32e264b89f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.588987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f0553-73d8-4814-85b8-a32e264b89f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.599730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f0553-73d8-4814-85b8-a32e264b89f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.601275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f0553-73d8-4814-85b8-a32e264b89f4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-391f0553-73d8-4814-85b8-a32e264b89f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:24.618934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:24.619900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:24.620792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:24.620792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:24.620792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:24.620792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:24.620792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 535/1067] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:24.626165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f0553-73d8-4814-85b8-a32e264b89f4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:24.650655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:24.674306 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a311b75-5aab-4a7d-9d5b-6dcee1d34c4d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.710296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b618a8f-b9a4-416d-ae42-2842bba92674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.711665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b618a8f-b9a4-416d-ae42-2842bba92674 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.711880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b618a8f-b9a4-416d-ae42-2842bba92674 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.712153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b618a8f-b9a4-416d-ae42-2842bba92674 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.712900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b618a8f-b9a4-416d-ae42-2842bba92674 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.714382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b618a8f-b9a4-416d-ae42-2842bba92674 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.731384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f0553-73d8-4814-85b8-a32e264b89f4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.734942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 533/1068] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Jul 2 03:22:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:24.778624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9937a4a-6262-4672-83f2-5f2852c2cda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.779213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9937a4a-6262-4672-83f2-5f2852c2cda6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.779526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9937a4a-6262-4672-83f2-5f2852c2cda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.779801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9937a4a-6262-4672-83f2-5f2852c2cda6 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.780529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9937a4a-6262-4672-83f2-5f2852c2cda6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:24.780837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9937a4a-6262-4672-83f2-5f2852c2cda6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.782269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9937a4a-6262-4672-83f2-5f2852c2cda6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.796294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 536/1069] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:24.801957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b618a8f-b9a4-416d-ae42-2842bba92674 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:24.837467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:24.917791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b618a8f-b9a4-416d-ae42-2842bba92674 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:24.953258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4690e-c232-4a10-8cf0-fa406d21a3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:24.954063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4690e-c232-4a10-8cf0-fa406d21a3ec None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:24.954343 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4690e-c232-4a10-8cf0-fa406d21a3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:24.954615 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4690e-c232-4a10-8cf0-fa406d21a3ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:24.961763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4690e-c232-4a10-8cf0-fa406d21a3ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:24.963227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4690e-c232-4a10-8cf0-fa406d21a3ec None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-62c4690e-c232-4a10-8cf0-fa406d21a3ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:22:24.977603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:24.978940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:24.979878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:22:24.979878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:22:24.979878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:22:24.979878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:22:24.979878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:22:24.979878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:22:24.979878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:24.979878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:24.979878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 534/1070] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:24.984577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9937a4a-6262-4672-83f2-5f2852c2cda6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:25.083192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:25.111080 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9937a4a-6262-4672-83f2-5f2852c2cda6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:25.151883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9ff3d-ded0-4f1d-8b12-af0f8de46c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:25.152536 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9ff3d-ded0-4f1d-8b12-af0f8de46c12 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:25.152774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9ff3d-ded0-4f1d-8b12-af0f8de46c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:25.153079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9ff3d-ded0-4f1d-8b12-af0f8de46c12 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:25.153858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9ff3d-ded0-4f1d-8b12-af0f8de46c12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:25.154164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9ff3d-ded0-4f1d-8b12-af0f8de46c12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:25.155617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9ff3d-ded0-4f1d-8b12-af0f8de46c12 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:25.168186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 537/1071] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:25.173955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4690e-c232-4a10-8cf0-fa406d21a3ec None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:25.210584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89847837-d62a-44ee-9600-ac523a2d8945 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:25.210991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89847837-d62a-44ee-9600-ac523a2d8945 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:25.211269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89847837-d62a-44ee-9600-ac523a2d8945 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:25.213006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 538/1072] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:25.218073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0fea6-8288-45ea-a0c6-c06fcdfc3420 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:25.218357 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0fea6-8288-45ea-a0c6-c06fcdfc3420 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:25.218612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0fea6-8288-45ea-a0c6-c06fcdfc3420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:25.236970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c4f0fea6-8288-45ea-a0c6-c06fcdfc3420 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:25.238147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c4f0fea6-8288-45ea-a0c6-c06fcdfc3420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:25.271901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:25.286560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4690e-c232-4a10-8cf0-fa406d21a3ec None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:25.309032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 539/1073] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:25.321888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8b344-338d-408f-a34d-48dcae5ac705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:25.322789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8b344-338d-408f-a34d-48dcae5ac705 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:25.323037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8b344-338d-408f-a34d-48dcae5ac705 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:25.323412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8b344-338d-408f-a34d-48dcae5ac705 None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/8b7bc20034974816905978dd365993f8/roles/d5b0722ac98a4453bc506335e1e71cab` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:25.325794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8b344-338d-408f-a34d-48dcae5ac705 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=8b7bc20034974816905978dd365993f8, role_id=d5b0722ac98a4453bc506335e1e71cab)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:25.327241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8b344-338d-408f-a34d-48dcae5ac705 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:25.333749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0fea6-8288-45ea-a0c6-c06fcdfc3420 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:25.347330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 535/1074] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:22:25 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/8b7bc20034974816905978dd365993f8/roles/d5b0722ac98a4453bc506335e1e71cab => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:22:25.419623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:25.457821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0fea6-8288-45ea-a0c6-c06fcdfc3420 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:25.497289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1453899-187e-4492-a141-3b98b832f21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:25.498084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1453899-187e-4492-a141-3b98b832f21c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:25.498369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1453899-187e-4492-a141-3b98b832f21c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:25.498644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1453899-187e-4492-a141-3b98b832f21c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:25.512665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1453899-187e-4492-a141-3b98b832f21c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-c1453899-187e-4492-a141-3b98b832f21c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:25.520304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:25.521238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:25.522288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 540/1075] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:22:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:25.528316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8b344-338d-408f-a34d-48dcae5ac705 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:25.630390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8b344-338d-408f-a34d-48dcae5ac705 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:25.665166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376c112-8c29-484f-a468-4728c4ec90a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:25.665836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376c112-8c29-484f-a468-4728c4ec90a3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:25.666120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376c112-8c29-484f-a468-4728c4ec90a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:25.666395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376c112-8c29-484f-a468-4728c4ec90a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:25.667136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376c112-8c29-484f-a468-4728c4ec90a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:25.667471 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376c112-8c29-484f-a468-4728c4ec90a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:25.668850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376c112-8c29-484f-a468-4728c4ec90a3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:25.677474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 536/1076] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:22:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:25.684347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1453899-187e-4492-a141-3b98b832f21c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:25.787919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1453899-187e-4492-a141-3b98b832f21c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:25.815825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:25.823377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06f8d7-f2b7-4268-a196-b414b1f9f5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:25.824319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f8d7-f2b7-4268-a196-b414b1f9f5ca None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:25.824646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f8d7-f2b7-4268-a196-b414b1f9f5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:25.824920 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f8d7-f2b7-4268-a196-b414b1f9f5ca None admin] PATH_INFO: `/v3/users/6e96456378b14f33a85ec4804fe8488e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:25.826121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06f8d7-f2b7-4268-a196-b414b1f9f5ca None admin] RBAC: Authorizing `identity:get_user(user_id=6e96456378b14f33a85ec4804fe8488e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:25.827605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06f8d7-f2b7-4268-a196-b414b1f9f5ca None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:25.829389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 541/1077] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:25 2022] GET /identity/v3/users/6e96456378b14f33a85ec4804fe8488e => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:25.835504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376c112-8c29-484f-a468-4728c4ec90a3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:25.850941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:25.942470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376c112-8c29-484f-a468-4728c4ec90a3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:25.975222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb7dcb-a1a5-416c-bb01-4fc947703b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:25.976109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb7dcb-a1a5-416c-bb01-4fc947703b00 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:25.976430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb7dcb-a1a5-416c-bb01-4fc947703b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:25.976688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb7dcb-a1a5-416c-bb01-4fc947703b00 None admin] PATH_INFO: `/v3/projects/0fd344da8a9648ec8c2e2b28593e8708` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:25.978015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb7dcb-a1a5-416c-bb01-4fc947703b00 None admin] RBAC: Authorizing `identity:get_project(project_id=0fd344da8a9648ec8c2e2b28593e8708)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:25.979480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb7dcb-a1a5-416c-bb01-4fc947703b00 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:25.981351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 537/1078] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:22:25 2022] GET /identity/v3/projects/0fd344da8a9648ec8c2e2b28593e8708 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:25.987107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06f8d7-f2b7-4268-a196-b414b1f9f5ca None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:26.090014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06f8d7-f2b7-4268-a196-b414b1f9f5ca None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:26.130785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc24458-256a-44ff-b3d7-d176b131bc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:26.131727 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc24458-256a-44ff-b3d7-d176b131bc4b None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:26.132050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc24458-256a-44ff-b3d7-d176b131bc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:26.132326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc24458-256a-44ff-b3d7-d176b131bc4b None admin] PATH_INFO: `/v3/projects/0fd344da8a9648ec8c2e2b28593e8708/users/6e96456378b14f33a85ec4804fe8488e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:26.134541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc24458-256a-44ff-b3d7-d176b131bc4b None admin] RBAC: Authorizing `identity:create_grant(project_id=0fd344da8a9648ec8c2e2b28593e8708, user_id=6e96456378b14f33a85ec4804fe8488e, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:26.136063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc24458-256a-44ff-b3d7-d176b131bc4b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:26.150675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 542/1079] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:22:25 2022] PUT /identity/v3/projects/0fd344da8a9648ec8c2e2b28593e8708/users/6e96456378b14f33a85ec4804fe8488e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:22:26.781596 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148f832-7160-4a8b-bab8-194cf158f038 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:26.781849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148f832-7160-4a8b-bab8-194cf158f038 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:26.782123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148f832-7160-4a8b-bab8-194cf158f038 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:26.783723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 538/1080] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:26.788724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ae506-42f8-49a8-8d41-f293c3be85a3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:26.789198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ae506-42f8-49a8-8d41-f293c3be85a3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:26.789539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ae506-42f8-49a8-8d41-f293c3be85a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:26.807441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-156ae506-42f8-49a8-8d41-f293c3be85a3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:26.808612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-156ae506-42f8-49a8-8d41-f293c3be85a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:26.866132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:26.881510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 543/1081] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:26.903568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148f832-7160-4a8b-bab8-194cf158f038 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:26.989199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:27.012588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148f832-7160-4a8b-bab8-194cf158f038 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:27.049886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53217d9f-0cd6-43a3-ace7-ba75a5ac2641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:27.050453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53217d9f-0cd6-43a3-ace7-ba75a5ac2641 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:27.050747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53217d9f-0cd6-43a3-ace7-ba75a5ac2641 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:27.050978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53217d9f-0cd6-43a3-ace7-ba75a5ac2641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:27.125696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53217d9f-0cd6-43a3-ace7-ba75a5ac2641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:27.128955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53217d9f-0cd6-43a3-ace7-ba75a5ac2641 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:27.213957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:27.237942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 539/1082] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:22:27.243903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ae506-42f8-49a8-8d41-f293c3be85a3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:27.348736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ae506-42f8-49a8-8d41-f293c3be85a3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:27.383313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21685a19-7433-489c-b4a8-81bdfc363638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:27.384125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21685a19-7433-489c-b4a8-81bdfc363638 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:27.384400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21685a19-7433-489c-b4a8-81bdfc363638 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:27.384666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21685a19-7433-489c-b4a8-81bdfc363638 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:27.394864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21685a19-7433-489c-b4a8-81bdfc363638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:27.395853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db758f-6259-4dde-be5c-f04936e40000 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:27.396156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db758f-6259-4dde-be5c-f04936e40000 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:27.396334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21685a19-7433-489c-b4a8-81bdfc363638 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:27.396428 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db758f-6259-4dde-be5c-f04936e40000 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:27.399138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 540/1083] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:22:27.403041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29434551-be2b-4c2b-9eb1-02bd90a7b0c3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:27.403354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29434551-be2b-4c2b-9eb1-02bd90a7b0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:27.403637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29434551-be2b-4c2b-9eb1-02bd90a7b0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-21685a19-7433-489c-b4a8-81bdfc363638 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:27.415830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:27.416724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:27.422801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:27.422801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:27.422801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:27.422801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:27.422801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 544/1084] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:27.423088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21685a19-7433-489c-b4a8-81bdfc363638 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:27.423159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-29434551-be2b-4c2b-9eb1-02bd90a7b0c3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:27.424313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29434551-be2b-4c2b-9eb1-02bd90a7b0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:27.470864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:27.504865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 541/1085] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:27.528124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29434551-be2b-4c2b-9eb1-02bd90a7b0c3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:27.532782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21685a19-7433-489c-b4a8-81bdfc363638 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:27.567478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a51b-650d-4209-a5f8-1612ae7dee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:27.568142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a51b-650d-4209-a5f8-1612ae7dee94 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:27.568427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a51b-650d-4209-a5f8-1612ae7dee94 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:27.568700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a51b-650d-4209-a5f8-1612ae7dee94 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:27.569418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2a51b-650d-4209-a5f8-1612ae7dee94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:27.569720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2a51b-650d-4209-a5f8-1612ae7dee94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:27.571349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2a51b-650d-4209-a5f8-1612ae7dee94 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:27.582784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 545/1086] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:27.649712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29434551-be2b-4c2b-9eb1-02bd90a7b0c3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:27.662088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:27.690721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd5d10-92f0-4310-83c8-070c1b6790bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:27.691578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd5d10-92f0-4310-83c8-070c1b6790bc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:27.691866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd5d10-92f0-4310-83c8-070c1b6790bc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:27.692141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd5d10-92f0-4310-83c8-070c1b6790bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:27.704026 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd5d10-92f0-4310-83c8-070c1b6790bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-65dd5d10-92f0-4310-83c8-070c1b6790bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:27.712505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:27.713505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:27.719494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 542/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Sat Jul 2 03:22:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:27.720832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a51b-650d-4209-a5f8-1612ae7dee94 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:27.826356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:27.830251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a51b-650d-4209-a5f8-1612ae7dee94 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:27.866844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141839d9-f3a8-4d46-8b29-0f0089f1a5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:27.867757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141839d9-f3a8-4d46-8b29-0f0089f1a5dc None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:27.868047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141839d9-f3a8-4d46-8b29-0f0089f1a5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:27.868322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141839d9-f3a8-4d46-8b29-0f0089f1a5dc None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:27.869071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141839d9-f3a8-4d46-8b29-0f0089f1a5dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:27.869380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141839d9-f3a8-4d46-8b29-0f0089f1a5dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:27.870763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141839d9-f3a8-4d46-8b29-0f0089f1a5dc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:27.879658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 546/1088] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Jul 2 03:22:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:27.885333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd5d10-92f0-4310-83c8-070c1b6790bc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:27.984755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd5d10-92f0-4310-83c8-070c1b6790bc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.019010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018e95a-48c0-410f-92e5-2f2785ff6116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.019852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018e95a-48c0-410f-92e5-2f2785ff6116 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.020133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018e95a-48c0-410f-92e5-2f2785ff6116 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.020403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018e95a-48c0-410f-92e5-2f2785ff6116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.031414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018e95a-48c0-410f-92e5-2f2785ff6116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:28.032859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018e95a-48c0-410f-92e5-2f2785ff6116 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7018e95a-48c0-410f-92e5-2f2785ff6116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:28.050208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:28.051150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:28.052169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:28.052169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:28.052169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:28.052169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:28.052169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 543/1089] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:28.057776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141839d9-f3a8-4d46-8b29-0f0089f1a5dc None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:28.162212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141839d9-f3a8-4d46-8b29-0f0089f1a5dc None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.164224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f7e41-c049-49e6-ad7a-beeb7b589d8d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.164520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f7e41-c049-49e6-ad7a-beeb7b589d8d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.164859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f7e41-c049-49e6-ad7a-beeb7b589d8d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.166263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 544/1090] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:28.171326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd9386-147f-47be-ab03-b99d139f88e5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.171598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd9386-147f-47be-ab03-b99d139f88e5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.171872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd9386-147f-47be-ab03-b99d139f88e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.189643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-94fd9386-147f-47be-ab03-b99d139f88e5 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:28.190837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-94fd9386-147f-47be-ab03-b99d139f88e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:28.196216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70314e2-2562-46f1-989a-8d338e59939f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.196875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70314e2-2562-46f1-989a-8d338e59939f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.197154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70314e2-2562-46f1-989a-8d338e59939f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.197424 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70314e2-2562-46f1-989a-8d338e59939f None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.198132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70314e2-2562-46f1-989a-8d338e59939f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:28.198434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70314e2-2562-46f1-989a-8d338e59939f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:28.199830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70314e2-2562-46f1-989a-8d338e59939f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:28.209359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:28.212207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 547/1091] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:28.218072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70314e2-2562-46f1-989a-8d338e59939f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:28.242105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:28.263127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 545/1092] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:28.286761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd9386-147f-47be-ab03-b99d139f88e5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:28.321681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70314e2-2562-46f1-989a-8d338e59939f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.355803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff45ff-12ca-4730-a55f-2df3dbd48735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.356585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff45ff-12ca-4730-a55f-2df3dbd48735 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.356871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff45ff-12ca-4730-a55f-2df3dbd48735 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.357147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff45ff-12ca-4730-a55f-2df3dbd48735 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.364651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff45ff-12ca-4730-a55f-2df3dbd48735 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:28.366154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff45ff-12ca-4730-a55f-2df3dbd48735 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-dfff45ff-12ca-4730-a55f-2df3dbd48735 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:22:28.380556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:28.381630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 02 03:22:28.382745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:28.385027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 548/1093] 10.222.0.42 () {56 vars in 1093 bytes} [Sat Jul 2 03:22:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:28.387662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd9386-147f-47be-ab03-b99d139f88e5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.390318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff45ff-12ca-4730-a55f-2df3dbd48735 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:28.421290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ca10d-9c81-478b-b410-f3cc050e5ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.422081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ca10d-9c81-478b-b410-f3cc050e5ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.422366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ca10d-9c81-478b-b410-f3cc050e5ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.422647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ca10d-9c81-478b-b410-f3cc050e5ab9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.432805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ca10d-9c81-478b-b410-f3cc050e5ab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e90ca10d-9c81-478b-b410-f3cc050e5ab9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:28.440449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:28.441455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:28.442310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 546/1094] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:28.447671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ca10d-9c81-478b-b410-f3cc050e5ab9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:28.463025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:28.473032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:28.493174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff45ff-12ca-4730-a55f-2df3dbd48735 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.526960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c81a5a-9cdd-4a03-98e1-a03da62eff21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.527650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c81a5a-9cdd-4a03-98e1-a03da62eff21 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.527933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c81a5a-9cdd-4a03-98e1-a03da62eff21 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.528205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c81a5a-9cdd-4a03-98e1-a03da62eff21 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.529060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c81a5a-9cdd-4a03-98e1-a03da62eff21 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:28.529368 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c81a5a-9cdd-4a03-98e1-a03da62eff21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:28.530802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c81a5a-9cdd-4a03-98e1-a03da62eff21 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:28.548155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 549/1095] 10.222.0.42 () {56 vars in 1133 bytes} [Sat Jul 2 03:22:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:28.552088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ca10d-9c81-478b-b410-f3cc050e5ab9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.553970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c81a5a-9cdd-4a03-98e1-a03da62eff21 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:28.586401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b039518-2e02-4f1a-ab31-85057b7b2483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.587148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b039518-2e02-4f1a-ab31-85057b7b2483 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.587376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b039518-2e02-4f1a-ab31-85057b7b2483 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.587658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b039518-2e02-4f1a-ab31-85057b7b2483 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.588427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b039518-2e02-4f1a-ab31-85057b7b2483 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:28.588738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b039518-2e02-4f1a-ab31-85057b7b2483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:28.590120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b039518-2e02-4f1a-ab31-85057b7b2483 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:28.599662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 547/1096] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:28.605384 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b039518-2e02-4f1a-ab31-85057b7b2483 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:28.650874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c81a5a-9cdd-4a03-98e1-a03da62eff21 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.675773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:28.684196 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71074fac-cd3e-4d19-920d-d8a50fcc07ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.684865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71074fac-cd3e-4d19-920d-d8a50fcc07ae None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.685149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71074fac-cd3e-4d19-920d-d8a50fcc07ae None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.685431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71074fac-cd3e-4d19-920d-d8a50fcc07ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.695943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:28.701188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b039518-2e02-4f1a-ab31-85057b7b2483 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.737431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafaeb0-016e-40e4-afcb-fd1b6b33b4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.738296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafaeb0-016e-40e4-afcb-fd1b6b33b4ce None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.738576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafaeb0-016e-40e4-afcb-fd1b6b33b4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.738852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafaeb0-016e-40e4-afcb-fd1b6b33b4ce None admin] PATH_INFO: `/v3/users/6e96456378b14f33a85ec4804fe8488e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.740387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafaeb0-016e-40e4-afcb-fd1b6b33b4ce None admin] RBAC: Authorizing `identity:get_user(user_id=6e96456378b14f33a85ec4804fe8488e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:28.741870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafaeb0-016e-40e4-afcb-fd1b6b33b4ce None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:28.743868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 548/1097] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:28 2022] GET /identity/v3/users/6e96456378b14f33a85ec4804fe8488e => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:28.749518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafaeb0-016e-40e4-afcb-fd1b6b33b4ce None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:28.755327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71074fac-cd3e-4d19-920d-d8a50fcc07ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:28.756793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71074fac-cd3e-4d19-920d-d8a50fcc07ae None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:28.850498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafaeb0-016e-40e4-afcb-fd1b6b33b4ce None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.854992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 550/1098] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 02 03:22:28.861594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71074fac-cd3e-4d19-920d-d8a50fcc07ae None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:28.886955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714af6d6-b81d-47cb-a689-b8f928188fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.887641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714af6d6-b81d-47cb-a689-b8f928188fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.887924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714af6d6-b81d-47cb-a689-b8f928188fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.888205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714af6d6-b81d-47cb-a689-b8f928188fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:28.900074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:28.905040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:28.956226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714af6d6-b81d-47cb-a689-b8f928188fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:28.957703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714af6d6-b81d-47cb-a689-b8f928188fd9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:28.963045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71074fac-cd3e-4d19-920d-d8a50fcc07ae None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:28.998496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570de7a-1b51-443f-ba7b-85d16f98deba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:28.999207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570de7a-1b51-443f-ba7b-85d16f98deba None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:28.999509 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570de7a-1b51-443f-ba7b-85d16f98deba None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:28.999779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570de7a-1b51-443f-ba7b-85d16f98deba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:29.010012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570de7a-1b51-443f-ba7b-85d16f98deba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:29.011504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570de7a-1b51-443f-ba7b-85d16f98deba None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b570de7a-1b51-443f-ba7b-85d16f98deba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:29.029113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:29.030216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:29.031221 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:22:29.031221 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:29.031221 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:22:29.031221 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:29.031221 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 551/1099] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:29.035973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570de7a-1b51-443f-ba7b-85d16f98deba None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:29.060861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 549/1100] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 02 03:22:29.062022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714af6d6-b81d-47cb-a689-b8f928188fd9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:29.103949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:29.124007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:29.136259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570de7a-1b51-443f-ba7b-85d16f98deba None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:29.166120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714af6d6-b81d-47cb-a689-b8f928188fd9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:29.174179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e0b38-8c6f-47ff-b2b0-873011d7ac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:29.174845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e0b38-8c6f-47ff-b2b0-873011d7ac43 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:29.175126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e0b38-8c6f-47ff-b2b0-873011d7ac43 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:29.175432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e0b38-8c6f-47ff-b2b0-873011d7ac43 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:29.176149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e0b38-8c6f-47ff-b2b0-873011d7ac43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:29.176453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e0b38-8c6f-47ff-b2b0-873011d7ac43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:29.177828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e0b38-8c6f-47ff-b2b0-873011d7ac43 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:29.189303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 552/1101] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:29.195433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e0b38-8c6f-47ff-b2b0-873011d7ac43 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:29.201967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1810fc7-82b8-424b-90e3-84c6e744facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:29.202833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1810fc7-82b8-424b-90e3-84c6e744facd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:29.203110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1810fc7-82b8-424b-90e3-84c6e744facd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:29.203434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1810fc7-82b8-424b-90e3-84c6e744facd None admin] PATH_INFO: `/v3/projects/0fd344da8a9648ec8c2e2b28593e8708` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:29.204607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1810fc7-82b8-424b-90e3-84c6e744facd None admin] RBAC: Authorizing `identity:get_project(project_id=0fd344da8a9648ec8c2e2b28593e8708)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:29.206046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1810fc7-82b8-424b-90e3-84c6e744facd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:29.207803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 550/1102] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:22:29 2022] GET /identity/v3/projects/0fd344da8a9648ec8c2e2b28593e8708 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:29.213846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1810fc7-82b8-424b-90e3-84c6e744facd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:29.291782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e0b38-8c6f-47ff-b2b0-873011d7ac43 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:29.310193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1810fc7-82b8-424b-90e3-84c6e744facd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:29.321846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:29.327550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db993557-d184-49fa-8c13-8f8b607b07aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:29.328349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db993557-d184-49fa-8c13-8f8b607b07aa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:29.328633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db993557-d184-49fa-8c13-8f8b607b07aa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:29.328913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db993557-d184-49fa-8c13-8f8b607b07aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:29.335760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db993557-d184-49fa-8c13-8f8b607b07aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:29.337803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db993557-d184-49fa-8c13-8f8b607b07aa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:29.344954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66beaea-c1e9-4b96-b87e-b594f705085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:29.345609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66beaea-c1e9-4b96-b87e-b594f705085c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:29.345893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66beaea-c1e9-4b96-b87e-b594f705085c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:29.346171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66beaea-c1e9-4b96-b87e-b594f705085c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:29.346997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66beaea-c1e9-4b96-b87e-b594f705085c None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=0fd344da8a9648ec8c2e2b28593e8708, user.id=6e96456378b14f33a85ec4804fe8488e) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:29.347333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66beaea-c1e9-4b96-b87e-b594f705085c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:29.348809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66beaea-c1e9-4b96-b87e-b594f705085c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-db993557-d184-49fa-8c13-8f8b607b07aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:22:29.351663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:29.352697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:29.353590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:22:29.353590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:22:29.353590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:22:29.353590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:22:29.353590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:22:29.353590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:22:29.353590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:22:29.353590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:29.353590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 553/1103] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:22:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:22:29.355205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 551/1104] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:29 2022] GET /identity/v3/role_assignments?user.id=6e96456378b14f33a85ec4804fe8488e&scope.project.id=0fd344da8a9648ec8c2e2b28593e8708&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:29.358680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db993557-d184-49fa-8c13-8f8b607b07aa None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:29.374227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:29.474621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db993557-d184-49fa-8c13-8f8b607b07aa None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:29.511234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b94f76-e7ad-4f57-b249-8e49bf73dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:29.511904 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b94f76-e7ad-4f57-b249-8e49bf73dc52 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:29.512181 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b94f76-e7ad-4f57-b249-8e49bf73dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:29.512440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b94f76-e7ad-4f57-b249-8e49bf73dc52 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:29.513259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b94f76-e7ad-4f57-b249-8e49bf73dc52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:29.513563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b94f76-e7ad-4f57-b249-8e49bf73dc52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:29.514976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b94f76-e7ad-4f57-b249-8e49bf73dc52 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:29.526798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 554/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:29.532955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66beaea-c1e9-4b96-b87e-b594f705085c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:29.634703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66beaea-c1e9-4b96-b87e-b594f705085c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:29.670620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a4b2b-f648-450d-a33f-594e31452bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:29.671278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a4b2b-f648-450d-a33f-594e31452bbf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:29.671589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a4b2b-f648-450d-a33f-594e31452bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:29.671866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a4b2b-f648-450d-a33f-594e31452bbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:29.672690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a4b2b-f648-450d-a33f-594e31452bbf None admin] RBAC: Adding query filter params (role.id=d5b0722ac98a4453bc506335e1e71cab, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=8b7bc20034974816905978dd365993f8) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:29.673003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a4b2b-f648-450d-a33f-594e31452bbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:29.674445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a4b2b-f648-450d-a33f-594e31452bbf None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:29.680756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 552/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:29 2022] GET /identity/v3/role_assignments?user.id=8b7bc20034974816905978dd365993f8&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5b0722ac98a4453bc506335e1e71cab => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:31.421219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae51ba-669e-411f-b6ae-055c3ad38e90 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:31.421743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae51ba-669e-411f-b6ae-055c3ad38e90 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:31.421743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae51ba-669e-411f-b6ae-055c3ad38e90 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:31.423043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 555/1107] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:31.429432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76097-6e45-45f5-b1c6-abde9605a0f1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:31.429958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76097-6e45-45f5-b1c6-abde9605a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:31.430242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc76097-6e45-45f5-b1c6-abde9605a0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:31.449601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ddc76097-6e45-45f5-b1c6-abde9605a0f1 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:31.450781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ddc76097-6e45-45f5-b1c6-abde9605a0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:31.482443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:31.520072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 553/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:31.544554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae51ba-669e-411f-b6ae-055c3ad38e90 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:31.621621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:31.664659 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae51ba-669e-411f-b6ae-055c3ad38e90 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:31.699824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b24f7-1a26-46d3-8996-fc0c9e201932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:31.700553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b24f7-1a26-46d3-8996-fc0c9e201932 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:31.700841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b24f7-1a26-46d3-8996-fc0c9e201932 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:31.701114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b24f7-1a26-46d3-8996-fc0c9e201932 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:31.702433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b24f7-1a26-46d3-8996-fc0c9e201932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:31.704016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b24f7-1a26-46d3-8996-fc0c9e201932 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:31.705634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 556/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:31.714855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76097-6e45-45f5-b1c6-abde9605a0f1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:31.821291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc76097-6e45-45f5-b1c6-abde9605a0f1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:31.856373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd23b88-c816-42f8-a5d6-8a1b903c226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:31.857016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23b88-c816-42f8-a5d6-8a1b903c226d None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:31.857431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23b88-c816-42f8-a5d6-8a1b903c226d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:31.857827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd23b88-c816-42f8-a5d6-8a1b903c226d None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:31.858574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd23b88-c816-42f8-a5d6-8a1b903c226d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:31.860212 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd23b88-c816-42f8-a5d6-8a1b903c226d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:31.881840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 554/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:22:31 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:34.173530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0693d-1fe2-4d86-aee7-8b0ba1b97d9e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:34.174101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0693d-1fe2-4d86-aee7-8b0ba1b97d9e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:34.174193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0693d-1fe2-4d86-aee7-8b0ba1b97d9e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:34.175850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 557/1111] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:34.182995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1362f3-2404-4260-945b-d12438cccbb1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:34.183674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1362f3-2404-4260-945b-d12438cccbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:34.183909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1362f3-2404-4260-945b-d12438cccbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:34.212743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9b1362f3-2404-4260-945b-d12438cccbb1 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:34.214418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9b1362f3-2404-4260-945b-d12438cccbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:34.238439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:34.299773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 555/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:22:34.326308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0693d-1fe2-4d86-aee7-8b0ba1b97d9e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:34.433150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0693d-1fe2-4d86-aee7-8b0ba1b97d9e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:34.465362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68adb1-4784-49b6-8ffe-23df7963be6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:34.466208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68adb1-4784-49b6-8ffe-23df7963be6d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:34.466487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68adb1-4784-49b6-8ffe-23df7963be6d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:34.466759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68adb1-4784-49b6-8ffe-23df7963be6d None admin] PATH_INFO: `/v3/roles/62ac4326d6d742c5b6684746953362b1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:34.468290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68adb1-4784-49b6-8ffe-23df7963be6d None admin] RBAC: Authorizing `identity:get_role(role_id=62ac4326d6d742c5b6684746953362b1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:34.469666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68adb1-4784-49b6-8ffe-23df7963be6d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:34.471027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 558/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:34 2022] GET /identity/v3/roles/62ac4326d6d742c5b6684746953362b1 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:34.477474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1362f3-2404-4260-945b-d12438cccbb1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:34.596493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1362f3-2404-4260-945b-d12438cccbb1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:34.633726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dd60a-b3a1-40b3-813d-fc2ab199fafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:34.634601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dd60a-b3a1-40b3-813d-fc2ab199fafa None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:34.634880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dd60a-b3a1-40b3-813d-fc2ab199fafa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:34.635150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dd60a-b3a1-40b3-813d-fc2ab199fafa None admin] PATH_INFO: `/v3/users/70d2e3b044fd4850bd23f8d96529f3d7` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:34.648070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78dd60a-b3a1-40b3-813d-fc2ab199fafa None admin] RBAC: Authorizing `identity:get_user(user_id=70d2e3b044fd4850bd23f8d96529f3d7)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:34.649584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78dd60a-b3a1-40b3-813d-fc2ab199fafa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:34.651260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 556/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:34 2022] GET /identity/v3/users/70d2e3b044fd4850bd23f8d96529f3d7 => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:34.660690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68adb1-4784-49b6-8ffe-23df7963be6d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:34.672465 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:34.764586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68adb1-4784-49b6-8ffe-23df7963be6d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:34.799335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6442d4-9702-4bc6-a0f5-accd320980fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:34.800244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6442d4-9702-4bc6-a0f5-accd320980fa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:34.800669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6442d4-9702-4bc6-a0f5-accd320980fa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:34.801075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6442d4-9702-4bc6-a0f5-accd320980fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:34.894284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6442d4-9702-4bc6-a0f5-accd320980fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:34.895784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6442d4-9702-4bc6-a0f5-accd320980fa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:34.905799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:34.999198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 559/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 02 03:22:35.005674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dd60a-b3a1-40b3-813d-fc2ab199fafa None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:35.022390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:35.118162 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dd60a-b3a1-40b3-813d-fc2ab199fafa None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:35.157476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d60ce-c3c1-4dd8-b10c-3d17474ed33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:35.158335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d60ce-c3c1-4dd8-b10c-3d17474ed33b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:35.158613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d60ce-c3c1-4dd8-b10c-3d17474ed33b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:35.158879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d60ce-c3c1-4dd8-b10c-3d17474ed33b None admin] PATH_INFO: `/v3/projects/0fd344da8a9648ec8c2e2b28593e8708` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:35.160092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d60ce-c3c1-4dd8-b10c-3d17474ed33b None admin] RBAC: Authorizing `identity:get_project(project_id=0fd344da8a9648ec8c2e2b28593e8708)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:35.161539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d60ce-c3c1-4dd8-b10c-3d17474ed33b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:35.163467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 557/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:22:35 2022] GET /identity/v3/projects/0fd344da8a9648ec8c2e2b28593e8708 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:35.170013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6442d4-9702-4bc6-a0f5-accd320980fa None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:35.279410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6442d4-9702-4bc6-a0f5-accd320980fa None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:35.289423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:35.318987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1172827-b314-439d-b08f-48ca151d1fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:35.319665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1172827-b314-439d-b08f-48ca151d1fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:35.319982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1172827-b314-439d-b08f-48ca151d1fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:35.320785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1172827-b314-439d-b08f-48ca151d1fd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:35.321701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1172827-b314-439d-b08f-48ca151d1fd2 None admin] RBAC: Adding query filter params (role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=0fd344da8a9648ec8c2e2b28593e8708, user.id=70d2e3b044fd4850bd23f8d96529f3d7) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:35.322018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1172827-b314-439d-b08f-48ca151d1fd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:35.323501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1172827-b314-439d-b08f-48ca151d1fd2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:35.329129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 560/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:35 2022] GET /identity/v3/role_assignments?user.id=70d2e3b044fd4850bd23f8d96529f3d7&scope.project.id=0fd344da8a9648ec8c2e2b28593e8708&role.id=62ac4326d6d742c5b6684746953362b1 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:37.337390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec11d5-0374-472d-bb80-499c0c76999a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:37.337978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec11d5-0374-472d-bb80-499c0c76999a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:37.337978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec11d5-0374-472d-bb80-499c0c76999a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:37.339317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 558/1118] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:37.345674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b6560-aa86-4e08-a844-335d43ad1ff6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:37.346303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b6560-aa86-4e08-a844-335d43ad1ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:37.346456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b6560-aa86-4e08-a844-335d43ad1ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:37.371839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e55b6560-aa86-4e08-a844-335d43ad1ff6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:37.373039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e55b6560-aa86-4e08-a844-335d43ad1ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:37.449646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 561/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:37.475702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec11d5-0374-472d-bb80-499c0c76999a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:37.561461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:37.582270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec11d5-0374-472d-bb80-499c0c76999a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:37.616190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c1f37-bc1e-452c-a497-2713d4166550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:37.617083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c1f37-bc1e-452c-a497-2713d4166550 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:37.617440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c1f37-bc1e-452c-a497-2713d4166550 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:37.617817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c1f37-bc1e-452c-a497-2713d4166550 None admin] PATH_INFO: `/v3/roles/62ac4326d6d742c5b6684746953362b1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:37.619432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c1f37-bc1e-452c-a497-2713d4166550 None admin] RBAC: Authorizing `identity:get_role(role_id=62ac4326d6d742c5b6684746953362b1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:37.620807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c1f37-bc1e-452c-a497-2713d4166550 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:37.622286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 559/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:37 2022] GET /identity/v3/roles/62ac4326d6d742c5b6684746953362b1 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:37.628845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b6560-aa86-4e08-a844-335d43ad1ff6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:37.758751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b6560-aa86-4e08-a844-335d43ad1ff6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:37.760811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:37.805829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba2e5a-c7b5-46b8-93ef-191021ecdffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:37.807336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba2e5a-c7b5-46b8-93ef-191021ecdffd None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:37.807730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba2e5a-c7b5-46b8-93ef-191021ecdffd None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:37.808138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba2e5a-c7b5-46b8-93ef-191021ecdffd None admin] PATH_INFO: `/v3/users/70d2e3b044fd4850bd23f8d96529f3d7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:37.809861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba2e5a-c7b5-46b8-93ef-191021ecdffd None admin] RBAC: Authorizing `identity:get_user(user_id=70d2e3b044fd4850bd23f8d96529f3d7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:37.811914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba2e5a-c7b5-46b8-93ef-191021ecdffd None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:37.814189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 562/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:37 2022] GET /identity/v3/users/70d2e3b044fd4850bd23f8d96529f3d7 => generated 286 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:37.822552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c1f37-bc1e-452c-a497-2713d4166550 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:37.928690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c1f37-bc1e-452c-a497-2713d4166550 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:37.962912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2cf44-007f-4d99-b00f-9f83eedb86ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:37.963796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2cf44-007f-4d99-b00f-9f83eedb86ec None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:37.964077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2cf44-007f-4d99-b00f-9f83eedb86ec None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:37.964352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2cf44-007f-4d99-b00f-9f83eedb86ec None admin] PATH_INFO: `/v3/projects/0fd344da8a9648ec8c2e2b28593e8708` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:37.965588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2cf44-007f-4d99-b00f-9f83eedb86ec None admin] RBAC: Authorizing `identity:get_project(project_id=0fd344da8a9648ec8c2e2b28593e8708)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:37.967041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2cf44-007f-4d99-b00f-9f83eedb86ec None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:37.968806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 560/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:22:37 2022] GET /identity/v3/projects/0fd344da8a9648ec8c2e2b28593e8708 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:37.975521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba2e5a-c7b5-46b8-93ef-191021ecdffd None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:38.089367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba2e5a-c7b5-46b8-93ef-191021ecdffd None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:38.126841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59960e19-3f6e-441e-b584-7fda635adae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:38.127776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59960e19-3f6e-441e-b584-7fda635adae2 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:38.128060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59960e19-3f6e-441e-b584-7fda635adae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:38.128335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59960e19-3f6e-441e-b584-7fda635adae2 None admin] PATH_INFO: `/v3/projects/0fd344da8a9648ec8c2e2b28593e8708/users/70d2e3b044fd4850bd23f8d96529f3d7/roles/62ac4326d6d742c5b6684746953362b1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:38.130759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59960e19-3f6e-441e-b584-7fda635adae2 None admin] RBAC: Authorizing `identity:create_grant(project_id=0fd344da8a9648ec8c2e2b28593e8708, user_id=70d2e3b044fd4850bd23f8d96529f3d7, role_id=62ac4326d6d742c5b6684746953362b1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:38.132240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59960e19-3f6e-441e-b584-7fda635adae2 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:38.149405 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 563/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:22:37 2022] PUT /identity/v3/projects/0fd344da8a9648ec8c2e2b28593e8708/users/70d2e3b044fd4850bd23f8d96529f3d7/roles/62ac4326d6d742c5b6684746953362b1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:22:40.127550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff53704-b528-4f0b-92f4-ba78c862ced2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:40.128254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff53704-b528-4f0b-92f4-ba78c862ced2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:40.128435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff53704-b528-4f0b-92f4-ba78c862ced2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:40.130599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 561/1124] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:40.136134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338734c5-bda8-49be-b851-fb6f30e268a3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:40.136564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338734c5-bda8-49be-b851-fb6f30e268a3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:40.136679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338734c5-bda8-49be-b851-fb6f30e268a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:40.155582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-338734c5-bda8-49be-b851-fb6f30e268a3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:40.156758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-338734c5-bda8-49be-b851-fb6f30e268a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:40.184071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:40.233433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 564/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:40.257908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff53704-b528-4f0b-92f4-ba78c862ced2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:40.287534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:40.377078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff53704-b528-4f0b-92f4-ba78c862ced2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:40.411738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f51cc-9a2b-45f0-9e4a-a3473b230001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:40.412588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f51cc-9a2b-45f0-9e4a-a3473b230001 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:40.412869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f51cc-9a2b-45f0-9e4a-a3473b230001 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:40.413153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f51cc-9a2b-45f0-9e4a-a3473b230001 None admin] PATH_INFO: `/v3/roles/62ac4326d6d742c5b6684746953362b1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:40.414767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f51cc-9a2b-45f0-9e4a-a3473b230001 None admin] RBAC: Authorizing `identity:get_role(role_id=62ac4326d6d742c5b6684746953362b1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:40.416202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f51cc-9a2b-45f0-9e4a-a3473b230001 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:40.417558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 562/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:40 2022] GET /identity/v3/roles/62ac4326d6d742c5b6684746953362b1 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:40.423705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338734c5-bda8-49be-b851-fb6f30e268a3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:40.527646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338734c5-bda8-49be-b851-fb6f30e268a3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:40.565577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c8bcb-15bc-4ab0-938c-7394b9f4424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:40.566922 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c8bcb-15bc-4ab0-938c-7394b9f4424a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:40.567377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c8bcb-15bc-4ab0-938c-7394b9f4424a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:40.567834 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c8bcb-15bc-4ab0-938c-7394b9f4424a None admin] PATH_INFO: `/v3/users/70d2e3b044fd4850bd23f8d96529f3d7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:40.569884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c8bcb-15bc-4ab0-938c-7394b9f4424a None admin] RBAC: Authorizing `identity:get_user(user_id=70d2e3b044fd4850bd23f8d96529f3d7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:40.572186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c8bcb-15bc-4ab0-938c-7394b9f4424a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:40.574739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 565/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:40 2022] GET /identity/v3/users/70d2e3b044fd4850bd23f8d96529f3d7 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:40.580800 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f51cc-9a2b-45f0-9e4a-a3473b230001 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:40.671653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:40.691373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f51cc-9a2b-45f0-9e4a-a3473b230001 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:40.728198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bafe93-2c91-4c93-836e-b34a598507b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:40.728841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bafe93-2c91-4c93-836e-b34a598507b0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:40.729117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bafe93-2c91-4c93-836e-b34a598507b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:40.729389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bafe93-2c91-4c93-836e-b34a598507b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:40.798569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bafe93-2c91-4c93-836e-b34a598507b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:40.800033 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bafe93-2c91-4c93-836e-b34a598507b0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:40.889642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:40.907162 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 563/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:22:40.913263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c8bcb-15bc-4ab0-938c-7394b9f4424a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:40.942173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:41.027534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c8bcb-15bc-4ab0-938c-7394b9f4424a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:41.062096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8ca43-6492-4c16-b363-ba44690746e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:41.062998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8ca43-6492-4c16-b363-ba44690746e1 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:41.063282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8ca43-6492-4c16-b363-ba44690746e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:41.063592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8ca43-6492-4c16-b363-ba44690746e1 None admin] PATH_INFO: `/v3/projects/0fd344da8a9648ec8c2e2b28593e8708` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:41.064776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8ca43-6492-4c16-b363-ba44690746e1 None admin] RBAC: Authorizing `identity:get_project(project_id=0fd344da8a9648ec8c2e2b28593e8708)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:41.066215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8ca43-6492-4c16-b363-ba44690746e1 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:41.067955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 566/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:22:40 2022] GET /identity/v3/projects/0fd344da8a9648ec8c2e2b28593e8708 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:41.074345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bafe93-2c91-4c93-836e-b34a598507b0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:41.172566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bafe93-2c91-4c93-836e-b34a598507b0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:41.209654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80878023-48b0-4bb1-b977-ed87c62acf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:41.210232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80878023-48b0-4bb1-b977-ed87c62acf34 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:41.210617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80878023-48b0-4bb1-b977-ed87c62acf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:41.210902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80878023-48b0-4bb1-b977-ed87c62acf34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:41.211828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80878023-48b0-4bb1-b977-ed87c62acf34 None admin] RBAC: Adding query filter params (role.id=62ac4326d6d742c5b6684746953362b1, scope.project.id=0fd344da8a9648ec8c2e2b28593e8708, user.id=70d2e3b044fd4850bd23f8d96529f3d7) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:41.212137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80878023-48b0-4bb1-b977-ed87c62acf34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:41.213605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80878023-48b0-4bb1-b977-ed87c62acf34 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:41.219608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 564/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:41 2022] GET /identity/v3/role_assignments?user.id=70d2e3b044fd4850bd23f8d96529f3d7&scope.project.id=0fd344da8a9648ec8c2e2b28593e8708&role.id=62ac4326d6d742c5b6684746953362b1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:22:43.263986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ea917-a311-4935-9db5-adfe3670d3bf None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:43.264824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ea917-a311-4935-9db5-adfe3670d3bf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:43.264824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ea917-a311-4935-9db5-adfe3670d3bf None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:43.267108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 567/1131] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:43.272649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022e60d-a28b-4bdc-9323-047e1504f65f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:43.273194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022e60d-a28b-4bdc-9323-047e1504f65f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:43.273194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022e60d-a28b-4bdc-9323-047e1504f65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:43.294609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4022e60d-a28b-4bdc-9323-047e1504f65f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:43.295830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4022e60d-a28b-4bdc-9323-047e1504f65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:43.342162 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:43.379142 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 565/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:43.403757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ea917-a311-4935-9db5-adfe3670d3bf None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:43.521414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:43.532761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ea917-a311-4935-9db5-adfe3670d3bf None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:43.576692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6de970-43fe-4091-9f35-b0ebf981d58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:43.577467 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de970-43fe-4091-9f35-b0ebf981d58a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:43.577751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de970-43fe-4091-9f35-b0ebf981d58a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:43.578027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6de970-43fe-4091-9f35-b0ebf981d58a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:43.579236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6de970-43fe-4091-9f35-b0ebf981d58a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:43.580714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6de970-43fe-4091-9f35-b0ebf981d58a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:43.582327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 568/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:43.588716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022e60d-a28b-4bdc-9323-047e1504f65f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:43.696766 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022e60d-a28b-4bdc-9323-047e1504f65f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:43.731523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d790b2-699a-4baa-9d24-fc5c4819e69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:43.732158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d790b2-699a-4baa-9d24-fc5c4819e69e None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:43.732436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d790b2-699a-4baa-9d24-fc5c4819e69e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:43.732671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d790b2-699a-4baa-9d24-fc5c4819e69e None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:43.733394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d790b2-699a-4baa-9d24-fc5c4819e69e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:43.734825 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d790b2-699a-4baa-9d24-fc5c4819e69e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:43.765783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 566/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:22:43 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:45.885061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28768fec-1ce1-46de-bdee-d553535ba85b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:45.885948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28768fec-1ce1-46de-bdee-d553535ba85b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:45.885948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28768fec-1ce1-46de-bdee-d553535ba85b None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:45.888167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 569/1135] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:45.894356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f753c3-20cc-4874-bd2c-afd5e847dba1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:45.895096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f753c3-20cc-4874-bd2c-afd5e847dba1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:45.895263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f753c3-20cc-4874-bd2c-afd5e847dba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:45.914090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-00f753c3-20cc-4874-bd2c-afd5e847dba1 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:45.915250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-00f753c3-20cc-4874-bd2c-afd5e847dba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:45.940581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:45.987038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 567/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:46.012094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28768fec-1ce1-46de-bdee-d553535ba85b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:46.122091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28768fec-1ce1-46de-bdee-d553535ba85b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:46.158065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2917d73-4e18-4e4b-a8cd-7c374445c246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:46.158775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2917d73-4e18-4e4b-a8cd-7c374445c246 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:46.159057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2917d73-4e18-4e4b-a8cd-7c374445c246 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:46.159335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2917d73-4e18-4e4b-a8cd-7c374445c246 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:46.160582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2917d73-4e18-4e4b-a8cd-7c374445c246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:46.162030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2917d73-4e18-4e4b-a8cd-7c374445c246 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:46.163732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 570/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:22:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:46.170270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f753c3-20cc-4874-bd2c-afd5e847dba1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:46.303149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f753c3-20cc-4874-bd2c-afd5e847dba1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:46.338356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129357c-4e7e-45e2-839e-cc8430fc03e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:46.339015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129357c-4e7e-45e2-839e-cc8430fc03e0 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:46.339307 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129357c-4e7e-45e2-839e-cc8430fc03e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:46.339593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129357c-4e7e-45e2-839e-cc8430fc03e0 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:46.340289 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129357c-4e7e-45e2-839e-cc8430fc03e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:46.341721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129357c-4e7e-45e2-839e-cc8430fc03e0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:46.367358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 568/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:22:46 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:22:48.490165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d4a19-9aa7-4159-b75f-c1b27ad4f59f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:48.490989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d4a19-9aa7-4159-b75f-c1b27ad4f59f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:48.491116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d4a19-9aa7-4159-b75f-c1b27ad4f59f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:48.493347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 571/1139] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:48.499654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ac47-cde9-45e3-80f2-43b6852205b9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:48.500083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ac47-cde9-45e3-80f2-43b6852205b9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:48.500256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937ac47-cde9-45e3-80f2-43b6852205b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:48.521970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e937ac47-cde9-45e3-80f2-43b6852205b9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:48.523656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e937ac47-cde9-45e3-80f2-43b6852205b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:48.535155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:48.608342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 569/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:48.632201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d4a19-9aa7-4159-b75f-c1b27ad4f59f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:48.668163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:48.745463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d4a19-9aa7-4159-b75f-c1b27ad4f59f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:48.779816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3efde2-24b7-4131-9c1c-c4c23bc3e008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:48.780594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3efde2-24b7-4131-9c1c-c4c23bc3e008 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:48.780873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3efde2-24b7-4131-9c1c-c4c23bc3e008 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:48.781147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3efde2-24b7-4131-9c1c-c4c23bc3e008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:48.791541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3efde2-24b7-4131-9c1c-c4c23bc3e008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2c3efde2-24b7-4131-9c1c-c4c23bc3e008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:48.799011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:48.800158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:48.801374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 572/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:22:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:48.808262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ac47-cde9-45e3-80f2-43b6852205b9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:48.923969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937ac47-cde9-45e3-80f2-43b6852205b9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:48.963141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a7267-a647-4e86-8220-9195c53a3780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:48.963827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a7267-a647-4e86-8220-9195c53a3780 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:48.964109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a7267-a647-4e86-8220-9195c53a3780 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:48.964385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a7267-a647-4e86-8220-9195c53a3780 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:48.965128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a7267-a647-4e86-8220-9195c53a3780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:48.965436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a7267-a647-4e86-8220-9195c53a3780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:48.966812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a7267-a647-4e86-8220-9195c53a3780 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:48.975001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 570/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:22:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:48.981594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3efde2-24b7-4131-9c1c-c4c23bc3e008 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:49.084611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:49.098069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3efde2-24b7-4131-9c1c-c4c23bc3e008 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:49.141747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdba898-7890-421c-96f0-71e1fe95dc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:49.142618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba898-7890-421c-96f0-71e1fe95dc77 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:49.142902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba898-7890-421c-96f0-71e1fe95dc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:49.143186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdba898-7890-421c-96f0-71e1fe95dc77 None admin] PATH_INFO: `/v3/users/f6e9b1b8525e46d3a28e8873b742811a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:49.157113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdba898-7890-421c-96f0-71e1fe95dc77 None admin] RBAC: Authorizing `identity:get_user(user_id=f6e9b1b8525e46d3a28e8873b742811a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:49.158574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdba898-7890-421c-96f0-71e1fe95dc77 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:49.160292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 573/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:48 2022] GET /identity/v3/users/f6e9b1b8525e46d3a28e8873b742811a => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:49.166468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a7267-a647-4e86-8220-9195c53a3780 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:49.179094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:49.278574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a7267-a647-4e86-8220-9195c53a3780 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:49.317320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e835a65-06f2-4c3a-b7f8-17e74d3e8e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:49.317958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e835a65-06f2-4c3a-b7f8-17e74d3e8e1f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:49.318236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e835a65-06f2-4c3a-b7f8-17e74d3e8e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:49.318503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e835a65-06f2-4c3a-b7f8-17e74d3e8e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:49.387554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e835a65-06f2-4c3a-b7f8-17e74d3e8e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:49.388933 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e835a65-06f2-4c3a-b7f8-17e74d3e8e1f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:49.398300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:49.494705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 571/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:22:49.501510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdba898-7890-421c-96f0-71e1fe95dc77 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:49.612139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdba898-7890-421c-96f0-71e1fe95dc77 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:49.650083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d2e5b-7002-4192-bca6-4090b8d80d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:49.650865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d2e5b-7002-4192-bca6-4090b8d80d05 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:49.651149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d2e5b-7002-4192-bca6-4090b8d80d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:49.651463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d2e5b-7002-4192-bca6-4090b8d80d05 None admin] PATH_INFO: `/v3/projects/3a10a80ec6bf42c7861a01476e21246c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:49.652742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d2e5b-7002-4192-bca6-4090b8d80d05 None admin] RBAC: Authorizing `identity:get_project(project_id=3a10a80ec6bf42c7861a01476e21246c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:49.654182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d2e5b-7002-4192-bca6-4090b8d80d05 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:49.655902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 574/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:22:49 2022] GET /identity/v3/projects/3a10a80ec6bf42c7861a01476e21246c => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:49.662511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e835a65-06f2-4c3a-b7f8-17e74d3e8e1f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:49.768565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e835a65-06f2-4c3a-b7f8-17e74d3e8e1f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:49.778145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:49.804852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d346a-9869-40f9-a532-f7048b6c7beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:49.805499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d346a-9869-40f9-a532-f7048b6c7beb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:49.805777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d346a-9869-40f9-a532-f7048b6c7beb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:49.806047 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d346a-9869-40f9-a532-f7048b6c7beb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:49.807190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d346a-9869-40f9-a532-f7048b6c7beb None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=3a10a80ec6bf42c7861a01476e21246c, user.id=f6e9b1b8525e46d3a28e8873b742811a) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:49.807526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d346a-9869-40f9-a532-f7048b6c7beb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:49.809052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d346a-9869-40f9-a532-f7048b6c7beb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:49.814421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 572/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:49 2022] GET /identity/v3/role_assignments?user.id=f6e9b1b8525e46d3a28e8873b742811a&scope.project.id=3a10a80ec6bf42c7861a01476e21246c&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:51.808167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cb02c-0c37-477b-b789-bdfa9b115ea1 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:51.808761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cb02c-0c37-477b-b789-bdfa9b115ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:51.808761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cb02c-0c37-477b-b789-bdfa9b115ea1 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:51.810087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 575/1147] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:51.816466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536df4-2954-48c8-8f59-ef3963b44e81 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:51.817180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536df4-2954-48c8-8f59-ef3963b44e81 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:51.817372 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536df4-2954-48c8-8f59-ef3963b44e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:51.842734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-af536df4-2954-48c8-8f59-ef3963b44e81 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:51.843934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-af536df4-2954-48c8-8f59-ef3963b44e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:51.927334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 573/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:51.957384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cb02c-0c37-477b-b789-bdfa9b115ea1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:51.978977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:52.066531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cb02c-0c37-477b-b789-bdfa9b115ea1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:52.101811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5881d-7733-4954-b645-d642ea169bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:52.102747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5881d-7733-4954-b645-d642ea169bae None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:52.103033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5881d-7733-4954-b645-d642ea169bae None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:52.103338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5881d-7733-4954-b645-d642ea169bae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:52.114150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5881d-7733-4954-b645-d642ea169bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-4fe5881d-7733-4954-b645-d642ea169bae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:52.122035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:52.123089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:52.124275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:52.127690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 576/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:22:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:52.129473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536df4-2954-48c8-8f59-ef3963b44e81 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:52.232204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536df4-2954-48c8-8f59-ef3963b44e81 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:52.233768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:52.268063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9230b3d-1e86-49f6-b021-00422f28abbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:52.268713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9230b3d-1e86-49f6-b021-00422f28abbf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:52.268994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9230b3d-1e86-49f6-b021-00422f28abbf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:52.269267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9230b3d-1e86-49f6-b021-00422f28abbf None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:52.270010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9230b3d-1e86-49f6-b021-00422f28abbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:52.270318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9230b3d-1e86-49f6-b021-00422f28abbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:52.271710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9230b3d-1e86-49f6-b021-00422f28abbf None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:52.280900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 574/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:22:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:52.287050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5881d-7733-4954-b645-d642ea169bae None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:52.372336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:52.397732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5881d-7733-4954-b645-d642ea169bae None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:52.433090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efa98f-6a8e-448f-be21-6e943b9f9269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:52.433955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efa98f-6a8e-448f-be21-6e943b9f9269 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:52.434245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efa98f-6a8e-448f-be21-6e943b9f9269 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:52.434515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efa98f-6a8e-448f-be21-6e943b9f9269 None admin] PATH_INFO: `/v3/users/f6e9b1b8525e46d3a28e8873b742811a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:52.435843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efa98f-6a8e-448f-be21-6e943b9f9269 None admin] RBAC: Authorizing `identity:get_user(user_id=f6e9b1b8525e46d3a28e8873b742811a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:52.437299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10efa98f-6a8e-448f-be21-6e943b9f9269 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:52.438938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 577/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:52 2022] GET /identity/v3/users/f6e9b1b8525e46d3a28e8873b742811a => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:52.445128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9230b3d-1e86-49f6-b021-00422f28abbf None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:52.548130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9230b3d-1e86-49f6-b021-00422f28abbf None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:52.583004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9218e-582c-4042-9c3e-a6b66801801c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:52.583883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9218e-582c-4042-9c3e-a6b66801801c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:52.584165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9218e-582c-4042-9c3e-a6b66801801c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:52.584446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9218e-582c-4042-9c3e-a6b66801801c None admin] PATH_INFO: `/v3/projects/3a10a80ec6bf42c7861a01476e21246c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:52.585736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9218e-582c-4042-9c3e-a6b66801801c None admin] RBAC: Authorizing `identity:get_project(project_id=3a10a80ec6bf42c7861a01476e21246c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:52.587170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9218e-582c-4042-9c3e-a6b66801801c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:52.588904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 575/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:22:52 2022] GET /identity/v3/projects/3a10a80ec6bf42c7861a01476e21246c => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:52.595019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efa98f-6a8e-448f-be21-6e943b9f9269 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:52.695838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efa98f-6a8e-448f-be21-6e943b9f9269 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:52.730042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866919c-b4a6-4029-96c2-9f56b3ab6131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:52.730905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866919c-b4a6-4029-96c2-9f56b3ab6131 None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:52.731182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866919c-b4a6-4029-96c2-9f56b3ab6131 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:52.731482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866919c-b4a6-4029-96c2-9f56b3ab6131 None admin] PATH_INFO: `/v3/projects/3a10a80ec6bf42c7861a01476e21246c/users/f6e9b1b8525e46d3a28e8873b742811a/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:52.733512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866919c-b4a6-4029-96c2-9f56b3ab6131 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a10a80ec6bf42c7861a01476e21246c, user_id=f6e9b1b8525e46d3a28e8873b742811a, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:52.734928 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4866919c-b4a6-4029-96c2-9f56b3ab6131 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:52.754030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 578/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:22:52 2022] PUT /identity/v3/projects/3a10a80ec6bf42c7861a01476e21246c/users/f6e9b1b8525e46d3a28e8873b742811a/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:22:54.717496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845e1d-2e4c-4d77-b89e-170d0b3b1ceb None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:54.718383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845e1d-2e4c-4d77-b89e-170d0b3b1ceb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:54.718383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845e1d-2e4c-4d77-b89e-170d0b3b1ceb None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:54.720591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 576/1154] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:54.730248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496d351-eceb-4feb-ad36-ab5153a723b6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:54.730679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496d351-eceb-4feb-ad36-ab5153a723b6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:54.730856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496d351-eceb-4feb-ad36-ab5153a723b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:54.747618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:54.753249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6496d351-eceb-4feb-ad36-ab5153a723b6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:54.755119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6496d351-eceb-4feb-ad36-ab5153a723b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:54.826832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 579/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:54.856321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41845e1d-2e4c-4d77-b89e-170d0b3b1ceb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:54.904620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:54.977055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41845e1d-2e4c-4d77-b89e-170d0b3b1ceb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:55.011064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005e446-e169-4430-b238-cdc72873fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:55.011876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005e446-e169-4430-b238-cdc72873fa94 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:55.012154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005e446-e169-4430-b238-cdc72873fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:55.012425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8005e446-e169-4430-b238-cdc72873fa94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:55.023818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8005e446-e169-4430-b238-cdc72873fa94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8005e446-e169-4430-b238-cdc72873fa94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:22:55.031261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:55.032360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:22:55.033640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 577/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:22:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:55.040861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496d351-eceb-4feb-ad36-ab5153a723b6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:55.146314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496d351-eceb-4feb-ad36-ab5153a723b6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:55.181123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a831-8a33-4e0a-b541-4417e55fefec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:55.182089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a831-8a33-4e0a-b541-4417e55fefec None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:55.182534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a831-8a33-4e0a-b541-4417e55fefec None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:55.182805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a831-8a33-4e0a-b541-4417e55fefec None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:55.183569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15a831-8a33-4e0a-b541-4417e55fefec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:55.183871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15a831-8a33-4e0a-b541-4417e55fefec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:55.185236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15a831-8a33-4e0a-b541-4417e55fefec None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:55.187446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:55.194982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 580/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:22:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:55.201690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005e446-e169-4430-b238-cdc72873fa94 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:55.303895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:55.314049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8005e446-e169-4430-b238-cdc72873fa94 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:55.348857 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc171f1-9be1-4071-9306-e131df6b5f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:55.349725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc171f1-9be1-4071-9306-e131df6b5f84 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:55.350007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc171f1-9be1-4071-9306-e131df6b5f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:55.350279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc171f1-9be1-4071-9306-e131df6b5f84 None admin] PATH_INFO: `/v3/users/f6e9b1b8525e46d3a28e8873b742811a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:55.351777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc171f1-9be1-4071-9306-e131df6b5f84 None admin] RBAC: Authorizing `identity:get_user(user_id=f6e9b1b8525e46d3a28e8873b742811a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:55.353229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc171f1-9be1-4071-9306-e131df6b5f84 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:55.354890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 578/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:22:55 2022] GET /identity/v3/users/f6e9b1b8525e46d3a28e8873b742811a => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:55.360926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a831-8a33-4e0a-b541-4417e55fefec None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:55.463937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15a831-8a33-4e0a-b541-4417e55fefec None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:55.499216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fd3cf-5185-4223-811b-c6fd65ad238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:55.499881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fd3cf-5185-4223-811b-c6fd65ad238a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:55.500167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fd3cf-5185-4223-811b-c6fd65ad238a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:55.500441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fd3cf-5185-4223-811b-c6fd65ad238a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:55.568378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:55.572162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fd3cf-5185-4223-811b-c6fd65ad238a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:55.573666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fd3cf-5185-4223-811b-c6fd65ad238a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:55.679627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 581/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:22:55.686424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc171f1-9be1-4071-9306-e131df6b5f84 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:55.790637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc171f1-9be1-4071-9306-e131df6b5f84 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:55.826030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338554e5-fa0d-468a-bde5-fc653fe345ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:55.826817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338554e5-fa0d-468a-bde5-fc653fe345ed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:55.827086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338554e5-fa0d-468a-bde5-fc653fe345ed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:55.827370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338554e5-fa0d-468a-bde5-fc653fe345ed None admin] PATH_INFO: `/v3/projects/3a10a80ec6bf42c7861a01476e21246c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:55.828694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338554e5-fa0d-468a-bde5-fc653fe345ed None admin] RBAC: Authorizing `identity:get_project(project_id=3a10a80ec6bf42c7861a01476e21246c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:55.830128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338554e5-fa0d-468a-bde5-fc653fe345ed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:55.831873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 579/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:22:55 2022] GET /identity/v3/projects/3a10a80ec6bf42c7861a01476e21246c => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:55.838217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fd3cf-5185-4223-811b-c6fd65ad238a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:55.944255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:55.949237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fd3cf-5185-4223-811b-c6fd65ad238a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:55.984461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76587d5a-6c24-456b-b38c-6094447d13b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:55.985095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76587d5a-6c24-456b-b38c-6094447d13b8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:55.985373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76587d5a-6c24-456b-b38c-6094447d13b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:55.985654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76587d5a-6c24-456b-b38c-6094447d13b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:55.986468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76587d5a-6c24-456b-b38c-6094447d13b8 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=3a10a80ec6bf42c7861a01476e21246c, user.id=f6e9b1b8525e46d3a28e8873b742811a) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:55.986776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76587d5a-6c24-456b-b38c-6094447d13b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:55.988232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76587d5a-6c24-456b-b38c-6094447d13b8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:55.994751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 582/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:22:55 2022] GET /identity/v3/role_assignments?user.id=f6e9b1b8525e46d3a28e8873b742811a&scope.project.id=3a10a80ec6bf42c7861a01476e21246c&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:22:58.039748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f75f-20bf-468c-b2fc-5796c94a51da None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:58.040711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f75f-20bf-468c-b2fc-5796c94a51da None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:58.040711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f75f-20bf-468c-b2fc-5796c94a51da None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:58.042824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 580/1162] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:22:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:22:58.051460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200d09a-805f-4fde-a9a2-2102cae6bad2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:58.051881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200d09a-805f-4fde-a9a2-2102cae6bad2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:58.052063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200d09a-805f-4fde-a9a2-2102cae6bad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:58.077661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2200d09a-805f-4fde-a9a2-2102cae6bad2 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:22:58.078844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2200d09a-805f-4fde-a9a2-2102cae6bad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:22:58.156109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 583/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:22:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:22:58.179837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769f75f-20bf-468c-b2fc-5796c94a51da None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:58.226323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:58.303941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769f75f-20bf-468c-b2fc-5796c94a51da None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:58.338905 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957e9d7-3e3d-41ac-b169-dcacea2cb08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:58.339569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957e9d7-3e3d-41ac-b169-dcacea2cb08f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:58.339881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957e9d7-3e3d-41ac-b169-dcacea2cb08f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:58.340119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957e9d7-3e3d-41ac-b169-dcacea2cb08f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:58.407862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957e9d7-3e3d-41ac-b169-dcacea2cb08f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:58.409304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957e9d7-3e3d-41ac-b169-dcacea2cb08f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:58.436882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:58.513326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 581/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:22:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:22:58.519717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200d09a-805f-4fde-a9a2-2102cae6bad2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:58.616550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:58.627049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200d09a-805f-4fde-a9a2-2102cae6bad2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:58.662390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0db0e-0100-47b1-8fab-d3f3ded985c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:58.663175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0db0e-0100-47b1-8fab-d3f3ded985c8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:58.663489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0db0e-0100-47b1-8fab-d3f3ded985c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:58.663842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0db0e-0100-47b1-8fab-d3f3ded985c8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:58.670569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0db0e-0100-47b1-8fab-d3f3ded985c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:58.672056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0db0e-0100-47b1-8fab-d3f3ded985c8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-84f0db0e-0100-47b1-8fab-d3f3ded985c8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:22:58.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:22:58.687710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:22:58.688991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 584/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:22:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:22:58.696371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957e9d7-3e3d-41ac-b169-dcacea2cb08f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:22:58.812195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957e9d7-3e3d-41ac-b169-dcacea2cb08f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:22:58.842817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:22:58.851495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d4030-b0d1-4434-842e-8c0ec7a9ca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:22:58.852154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4030-b0d1-4434-842e-8c0ec7a9ca1a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:22:58.852437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4030-b0d1-4434-842e-8c0ec7a9ca1a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:22:58.852713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4030-b0d1-4434-842e-8c0ec7a9ca1a None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:22:58.853435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d4030-b0d1-4434-842e-8c0ec7a9ca1a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:22:58.853740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d4030-b0d1-4434-842e-8c0ec7a9ca1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:22:58.855123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d4030-b0d1-4434-842e-8c0ec7a9ca1a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:22:58.863617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 582/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Sat Jul 2 03:22:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:00.897178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608484f-d44a-4bb9-ac14-e502ac57d3ed None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:00.897760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608484f-d44a-4bb9-ac14-e502ac57d3ed None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:00.897760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608484f-d44a-4bb9-ac14-e502ac57d3ed None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:00.899186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 585/1167] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:23:00.905540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28559d9-4434-4026-b0c6-8e634ecf9467 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:00.906301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28559d9-4434-4026-b0c6-8e634ecf9467 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:00.906433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28559d9-4434-4026-b0c6-8e634ecf9467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:00.933039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e28559d9-4434-4026-b0c6-8e634ecf9467 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:23:00.934232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e28559d9-4434-4026-b0c6-8e634ecf9467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:23:01.006882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 583/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:23:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:23:01.041801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608484f-d44a-4bb9-ac14-e502ac57d3ed None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:01.155151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608484f-d44a-4bb9-ac14-e502ac57d3ed None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:01.196063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86633fd-8f66-4eba-ace6-ea6f5769012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:01.196710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86633fd-8f66-4eba-ace6-ea6f5769012b None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:01.196988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86633fd-8f66-4eba-ace6-ea6f5769012b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:01.197262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86633fd-8f66-4eba-ace6-ea6f5769012b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:01.198004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86633fd-8f66-4eba-ace6-ea6f5769012b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:01.199399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86633fd-8f66-4eba-ace6-ea6f5769012b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:01.217604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 586/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Sat Jul 2 03:23:01 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:23:03.305413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aa0a8-1cf4-4dd1-a58c-7199b2504f55 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:03.305960 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aa0a8-1cf4-4dd1-a58c-7199b2504f55 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:03.305960 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aa0a8-1cf4-4dd1-a58c-7199b2504f55 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:03.307691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 584/1170] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:23:03.313997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72967d0-d2d7-43a8-88c0-d4bdc0ccbf63 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:03.314428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72967d0-d2d7-43a8-88c0-d4bdc0ccbf63 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:03.314602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72967d0-d2d7-43a8-88c0-d4bdc0ccbf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:03.335211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b72967d0-d2d7-43a8-88c0-d4bdc0ccbf63 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:23:03.336421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b72967d0-d2d7-43a8-88c0-d4bdc0ccbf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:23:03.344246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:03.420964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 587/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:23:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:23:03.461775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aa0a8-1cf4-4dd1-a58c-7199b2504f55 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:03.580472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aa0a8-1cf4-4dd1-a58c-7199b2504f55 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:03.603616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:03.616698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c6732d-d5b2-4c7d-8102-145adc7a8496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:03.617583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6732d-d5b2-4c7d-8102-145adc7a8496 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:03.617853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6732d-d5b2-4c7d-8102-145adc7a8496 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:03.618118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6732d-d5b2-4c7d-8102-145adc7a8496 None admin] PATH_INFO: `/v3/domains/df97a24bfe104485a6ed962b4159366a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:03.628642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6732d-d5b2-4c7d-8102-145adc7a8496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=df97a24bfe104485a6ed962b4159366a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:03.630091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c6732d-d5b2-4c7d-8102-145adc7a8496 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:03.631963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 585/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Sat Jul 2 03:23:03 2022] GET /identity/v3/domains/df97a24bfe104485a6ed962b4159366a => generated 267 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:03.638729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72967d0-d2d7-43a8-88c0-d4bdc0ccbf63 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:03.746146 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72967d0-d2d7-43a8-88c0-d4bdc0ccbf63 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:03.781551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a87642-de1a-4d53-ab52-3e9a75c40b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:03.782198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a87642-de1a-4d53-ab52-3e9a75c40b0e None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:03.782481 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a87642-de1a-4d53-ab52-3e9a75c40b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:03.782755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a87642-de1a-4d53-ab52-3e9a75c40b0e None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:03.783516 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a87642-de1a-4d53-ab52-3e9a75c40b0e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:03.784961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a87642-de1a-4d53-ab52-3e9a75c40b0e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:03.799371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:03.815201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 588/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:23:03 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:23:05.867315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05886e-a4f7-4d7e-9bea-4890607d7f45 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:05.867983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05886e-a4f7-4d7e-9bea-4890607d7f45 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:05.867983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05886e-a4f7-4d7e-9bea-4890607d7f45 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:05.869305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 586/1174] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:23:05.875590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3774192-a0f2-4936-9652-e294c5e269f0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:05.876337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3774192-a0f2-4936-9652-e294c5e269f0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:05.876523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3774192-a0f2-4936-9652-e294c5e269f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:05.902630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a3774192-a0f2-4936-9652-e294c5e269f0 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:23:05.903862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a3774192-a0f2-4936-9652-e294c5e269f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:23:05.981783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 589/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:23:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:23:06.007664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05886e-a4f7-4d7e-9bea-4890607d7f45 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:06.123735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05886e-a4f7-4d7e-9bea-4890607d7f45 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:06.161229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbd6c5-c891-4fff-a50c-6f4985b25cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:06.162134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbd6c5-c891-4fff-a50c-6f4985b25cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:06.162418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbd6c5-c891-4fff-a50c-6f4985b25cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:06.162692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbd6c5-c891-4fff-a50c-6f4985b25cc0 None admin] PATH_INFO: `/v3/domains/df97a24bfe104485a6ed962b4159366a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:06.163996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbd6c5-c891-4fff-a50c-6f4985b25cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=df97a24bfe104485a6ed962b4159366a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:06.165449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbd6c5-c891-4fff-a50c-6f4985b25cc0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:06.167129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 587/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Sat Jul 2 03:23:06 2022] GET /identity/v3/domains/df97a24bfe104485a6ed962b4159366a => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:06.173242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3774192-a0f2-4936-9652-e294c5e269f0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:06.284140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3774192-a0f2-4936-9652-e294c5e269f0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:06.309550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:06.323261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29205a7-715e-4144-b6cd-e3cfd7e538cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:06.323950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29205a7-715e-4144-b6cd-e3cfd7e538cc None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:06.324229 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29205a7-715e-4144-b6cd-e3cfd7e538cc None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:06.324497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29205a7-715e-4144-b6cd-e3cfd7e538cc None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:06.325199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29205a7-715e-4144-b6cd-e3cfd7e538cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:06.326618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29205a7-715e-4144-b6cd-e3cfd7e538cc None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:06.348217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 590/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:23:06 2022] POST /identity/v3/users => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:23:08.389328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc995a-6062-4289-9c48-d33e9b64ac15 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:08.390172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc995a-6062-4289-9c48-d33e9b64ac15 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:08.390172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc995a-6062-4289-9c48-d33e9b64ac15 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:08.392320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 588/1178] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:23:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:23:08.397724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa683f-e0f8-4aaf-9687-e04ed338f41a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:08.398156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa683f-e0f8-4aaf-9687-e04ed338f41a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:08.398256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa683f-e0f8-4aaf-9687-e04ed338f41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:08.419973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-baaa683f-e0f8-4aaf-9687-e04ed338f41a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:23:08.421845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-baaa683f-e0f8-4aaf-9687-e04ed338f41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:23:08.502799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 591/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:23:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:23:08.526501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc995a-6062-4289-9c48-d33e9b64ac15 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:08.584279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:08.637514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc995a-6062-4289-9c48-d33e9b64ac15 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:08.671808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17684e-6674-4cb6-b9a8-283882b41328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:08.672533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17684e-6674-4cb6-b9a8-283882b41328 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:08.672817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17684e-6674-4cb6-b9a8-283882b41328 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:08.673094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17684e-6674-4cb6-b9a8-283882b41328 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:08.684827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17684e-6674-4cb6-b9a8-283882b41328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3c17684e-6674-4cb6-b9a8-283882b41328 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:23:08.692300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:23:08.693336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:23:08.694530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 589/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:23:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:23:08.700376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa683f-e0f8-4aaf-9687-e04ed338f41a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:08.811462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa683f-e0f8-4aaf-9687-e04ed338f41a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:08.825448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:08.850784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7d1da-fb2d-4959-8ec7-83f3b468bcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:08.851468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7d1da-fb2d-4959-8ec7-83f3b468bcaa None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:08.851753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7d1da-fb2d-4959-8ec7-83f3b468bcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:08.852022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7d1da-fb2d-4959-8ec7-83f3b468bcaa None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:08.852758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7d1da-fb2d-4959-8ec7-83f3b468bcaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:23:08.853069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7d1da-fb2d-4959-8ec7-83f3b468bcaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:08.854432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7d1da-fb2d-4959-8ec7-83f3b468bcaa None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:08.862668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 592/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:23:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:08.868937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17684e-6674-4cb6-b9a8-283882b41328 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:08.977928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17684e-6674-4cb6-b9a8-283882b41328 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:08.988272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:09.012416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bfdbb-4e02-46b7-8737-b40fb3d2d039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:09.013281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bfdbb-4e02-46b7-8737-b40fb3d2d039 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:09.013556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bfdbb-4e02-46b7-8737-b40fb3d2d039 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:09.013811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bfdbb-4e02-46b7-8737-b40fb3d2d039 None admin] PATH_INFO: `/v3/users/53af0d3c76004ff1824136fe851eac47` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:09.026882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bfdbb-4e02-46b7-8737-b40fb3d2d039 None admin] RBAC: Authorizing `identity:get_user(user_id=53af0d3c76004ff1824136fe851eac47)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:09.028349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bfdbb-4e02-46b7-8737-b40fb3d2d039 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:09.029980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 590/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:23:08 2022] GET /identity/v3/users/53af0d3c76004ff1824136fe851eac47 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:09.036543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7d1da-fb2d-4959-8ec7-83f3b468bcaa None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:09.149175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7d1da-fb2d-4959-8ec7-83f3b468bcaa None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:09.184736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844cda2-d164-4789-b225-ec6abdcb58ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:09.185378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844cda2-d164-4789-b225-ec6abdcb58ce None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:09.185661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844cda2-d164-4789-b225-ec6abdcb58ce None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:09.185938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844cda2-d164-4789-b225-ec6abdcb58ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:09.230780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:09.261535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844cda2-d164-4789-b225-ec6abdcb58ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:09.262996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844cda2-d164-4789-b225-ec6abdcb58ce None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:09.365171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 593/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:23:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:23:09.372244 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bfdbb-4e02-46b7-8737-b40fb3d2d039 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:09.477444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bfdbb-4e02-46b7-8737-b40fb3d2d039 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:09.516086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3514d6-3a9d-4b31-b53d-eb6acd228695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:09.516883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3514d6-3a9d-4b31-b53d-eb6acd228695 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:09.517170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3514d6-3a9d-4b31-b53d-eb6acd228695 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:09.517442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3514d6-3a9d-4b31-b53d-eb6acd228695 None admin] PATH_INFO: `/v3/projects/8981b1f921ba44138b96ad4b129730dc` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:09.518661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3514d6-3a9d-4b31-b53d-eb6acd228695 None admin] RBAC: Authorizing `identity:get_project(project_id=8981b1f921ba44138b96ad4b129730dc)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:09.520140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3514d6-3a9d-4b31-b53d-eb6acd228695 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:09.521839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 591/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:23:09 2022] GET /identity/v3/projects/8981b1f921ba44138b96ad4b129730dc => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:09.527993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844cda2-d164-4789-b225-ec6abdcb58ce None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:09.599337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:09.629526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844cda2-d164-4789-b225-ec6abdcb58ce None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:09.665680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202d2b9-17eb-4567-bfab-7679af451408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:09.666311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202d2b9-17eb-4567-bfab-7679af451408 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:09.666596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202d2b9-17eb-4567-bfab-7679af451408 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:09.666867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202d2b9-17eb-4567-bfab-7679af451408 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:09.667714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202d2b9-17eb-4567-bfab-7679af451408 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=8981b1f921ba44138b96ad4b129730dc, user.id=53af0d3c76004ff1824136fe851eac47) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:23:09.668027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202d2b9-17eb-4567-bfab-7679af451408 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:09.669456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202d2b9-17eb-4567-bfab-7679af451408 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:09.674971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 594/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:23:09 2022] GET /identity/v3/role_assignments?user.id=53af0d3c76004ff1824136fe851eac47&scope.project.id=8981b1f921ba44138b96ad4b129730dc&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:11.638631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338121b-e7a7-4fb3-8eda-a825d169988d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:11.639219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338121b-e7a7-4fb3-8eda-a825d169988d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:11.639219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338121b-e7a7-4fb3-8eda-a825d169988d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:11.640606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 592/1186] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:23:11.646884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e269d-33b8-45c6-9c88-a532f9975110 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:11.647762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e269d-33b8-45c6-9c88-a532f9975110 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:11.647989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e269d-33b8-45c6-9c88-a532f9975110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:11.670704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-581e269d-33b8-45c6-9c88-a532f9975110 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:23:11.671929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-581e269d-33b8-45c6-9c88-a532f9975110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:23:11.740316 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 595/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:23:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:23:11.762884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338121b-e7a7-4fb3-8eda-a825d169988d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:11.804072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:11.871755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338121b-e7a7-4fb3-8eda-a825d169988d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:11.918465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e471429-77c7-4106-9e22-a7a677473ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:11.919585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471429-77c7-4106-9e22-a7a677473ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:11.919998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471429-77c7-4106-9e22-a7a677473ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:11.920398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471429-77c7-4106-9e22-a7a677473ec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:11.934341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e471429-77c7-4106-9e22-a7a677473ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7e471429-77c7-4106-9e22-a7a677473ec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:23:11.944600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:23:11.945692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:23:11.946734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 593/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:23:11.954306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e269d-33b8-45c6-9c88-a532f9975110 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:12.018167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:12.057430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e269d-33b8-45c6-9c88-a532f9975110 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:12.092528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643775-6887-4433-bd51-7a13bc870052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:12.093182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643775-6887-4433-bd51-7a13bc870052 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:12.093464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643775-6887-4433-bd51-7a13bc870052 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:12.093738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b643775-6887-4433-bd51-7a13bc870052 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:12.094475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643775-6887-4433-bd51-7a13bc870052 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:23:12.094783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643775-6887-4433-bd51-7a13bc870052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:12.096186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b643775-6887-4433-bd51-7a13bc870052 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:12.108594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 596/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:23:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:12.115281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e471429-77c7-4106-9e22-a7a677473ec1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:12.212274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:12.219955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e471429-77c7-4106-9e22-a7a677473ec1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:12.266460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00df5ef-fc2e-42b1-b930-054049ce30ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:12.267342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00df5ef-fc2e-42b1-b930-054049ce30ce None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:12.267627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00df5ef-fc2e-42b1-b930-054049ce30ce None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:12.267903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00df5ef-fc2e-42b1-b930-054049ce30ce None admin] PATH_INFO: `/v3/users/53af0d3c76004ff1824136fe851eac47` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:12.269209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00df5ef-fc2e-42b1-b930-054049ce30ce None admin] RBAC: Authorizing `identity:get_user(user_id=53af0d3c76004ff1824136fe851eac47)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:12.270662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00df5ef-fc2e-42b1-b930-054049ce30ce None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:12.272393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 594/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:23:12 2022] GET /identity/v3/users/53af0d3c76004ff1824136fe851eac47 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:12.279588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643775-6887-4433-bd51-7a13bc870052 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:12.398556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b643775-6887-4433-bd51-7a13bc870052 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:12.428228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:12.438087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de58c8-a91d-43c1-bec7-21e0543baac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:12.438945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de58c8-a91d-43c1-bec7-21e0543baac6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:12.439230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de58c8-a91d-43c1-bec7-21e0543baac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:12.439529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de58c8-a91d-43c1-bec7-21e0543baac6 None admin] PATH_INFO: `/v3/projects/8981b1f921ba44138b96ad4b129730dc` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:12.440749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de58c8-a91d-43c1-bec7-21e0543baac6 None admin] RBAC: Authorizing `identity:get_project(project_id=8981b1f921ba44138b96ad4b129730dc)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:12.442182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de58c8-a91d-43c1-bec7-21e0543baac6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:12.443901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 597/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:23:12 2022] GET /identity/v3/projects/8981b1f921ba44138b96ad4b129730dc => generated 361 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:12.450061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00df5ef-fc2e-42b1-b930-054049ce30ce None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:12.561578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00df5ef-fc2e-42b1-b930-054049ce30ce None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:12.599427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1873e39-c64f-4f3a-92db-559a7097c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:12.600324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1873e39-c64f-4f3a-92db-559a7097c2ff None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:12.600603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1873e39-c64f-4f3a-92db-559a7097c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:12.600981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1873e39-c64f-4f3a-92db-559a7097c2ff None admin] PATH_INFO: `/v3/projects/8981b1f921ba44138b96ad4b129730dc/users/53af0d3c76004ff1824136fe851eac47/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:12.603176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1873e39-c64f-4f3a-92db-559a7097c2ff None admin] RBAC: Authorizing `identity:create_grant(project_id=8981b1f921ba44138b96ad4b129730dc, user_id=53af0d3c76004ff1824136fe851eac47, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:12.604665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1873e39-c64f-4f3a-92db-559a7097c2ff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:12.619855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 595/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:23:12 2022] PUT /identity/v3/projects/8981b1f921ba44138b96ad4b129730dc/users/53af0d3c76004ff1824136fe851eac47/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:23:14.599602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997c551-8339-4c7d-9717-983e66b332de None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:14.600226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997c551-8339-4c7d-9717-983e66b332de None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:14.600226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997c551-8339-4c7d-9717-983e66b332de None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:14.601629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 598/1193] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:23:14.608058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe57eb-a2c6-4be9-83c0-088c0929b45d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:14.608763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe57eb-a2c6-4be9-83c0-088c0929b45d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:14.608909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe57eb-a2c6-4be9-83c0-088c0929b45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:14.635697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9dbe57eb-a2c6-4be9-83c0-088c0929b45d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:23:14.636897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9dbe57eb-a2c6-4be9-83c0-088c0929b45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:23:14.654720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:14.720665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 596/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:23:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:23:14.744851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997c551-8339-4c7d-9717-983e66b332de None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:14.857428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997c551-8339-4c7d-9717-983e66b332de None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:14.892027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c754c-5e3b-4b28-bbb8-eb1bbbffcbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:14.892812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c754c-5e3b-4b28-bbb8-eb1bbbffcbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:14.893089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c754c-5e3b-4b28-bbb8-eb1bbbffcbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:14.893365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c754c-5e3b-4b28-bbb8-eb1bbbffcbd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:14.905332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c754c-5e3b-4b28-bbb8-eb1bbbffcbd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a66c754c-5e3b-4b28-bbb8-eb1bbbffcbd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:23:14.912836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:23:14.913885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:23:14.914748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 599/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:23:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:23:14.922859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe57eb-a2c6-4be9-83c0-088c0929b45d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:15.030790 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe57eb-a2c6-4be9-83c0-088c0929b45d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:15.075325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca7553-57e2-4eaf-bbc7-12bc2254fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:15.076006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca7553-57e2-4eaf-bbc7-12bc2254fcd8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:15.076287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca7553-57e2-4eaf-bbc7-12bc2254fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:15.076560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca7553-57e2-4eaf-bbc7-12bc2254fcd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:15.077306 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca7553-57e2-4eaf-bbc7-12bc2254fcd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:23:15.077614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca7553-57e2-4eaf-bbc7-12bc2254fcd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:15.078990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca7553-57e2-4eaf-bbc7-12bc2254fcd8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:15.087847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 597/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:15.093749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c754c-5e3b-4b28-bbb8-eb1bbbffcbd4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:15.152413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:15.199564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c754c-5e3b-4b28-bbb8-eb1bbbffcbd4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:15.236262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a283cb-7d17-4897-8767-bc5a03adce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:15.237113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a283cb-7d17-4897-8767-bc5a03adce4c None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:15.237394 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a283cb-7d17-4897-8767-bc5a03adce4c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:15.237659 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a283cb-7d17-4897-8767-bc5a03adce4c None admin] PATH_INFO: `/v3/users/53af0d3c76004ff1824136fe851eac47` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:15.238950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a283cb-7d17-4897-8767-bc5a03adce4c None admin] RBAC: Authorizing `identity:get_user(user_id=53af0d3c76004ff1824136fe851eac47)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:15.240411 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a283cb-7d17-4897-8767-bc5a03adce4c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:15.242048 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 600/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Sat Jul 2 03:23:15 2022] GET /identity/v3/users/53af0d3c76004ff1824136fe851eac47 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:15.251206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca7553-57e2-4eaf-bbc7-12bc2254fcd8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:15.273055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:15.360241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca7553-57e2-4eaf-bbc7-12bc2254fcd8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:15.394831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc841e-9b9f-493e-848e-ac1fd5c88c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:15.395498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc841e-9b9f-493e-848e-ac1fd5c88c22 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:15.395786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc841e-9b9f-493e-848e-ac1fd5c88c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:15.396058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc841e-9b9f-493e-848e-ac1fd5c88c22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:15.475253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc841e-9b9f-493e-848e-ac1fd5c88c22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:15.476973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbc841e-9b9f-493e-848e-ac1fd5c88c22 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:15.493974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:15.580867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 598/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:23:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:23:15.588534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a283cb-7d17-4897-8767-bc5a03adce4c None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:15.692962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a283cb-7d17-4897-8767-bc5a03adce4c None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:15.716857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:15.735814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c8b09-8ca5-4b98-a457-c90bb54cfaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:15.737116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c8b09-8ca5-4b98-a457-c90bb54cfaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:15.737530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c8b09-8ca5-4b98-a457-c90bb54cfaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:15.737915 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c8b09-8ca5-4b98-a457-c90bb54cfaf8 None admin] PATH_INFO: `/v3/projects/8981b1f921ba44138b96ad4b129730dc` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:15.739960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c8b09-8ca5-4b98-a457-c90bb54cfaf8 None admin] RBAC: Authorizing `identity:get_project(project_id=8981b1f921ba44138b96ad4b129730dc)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:15.741909 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c8b09-8ca5-4b98-a457-c90bb54cfaf8 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:15.743644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 601/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Sat Jul 2 03:23:15 2022] GET /identity/v3/projects/8981b1f921ba44138b96ad4b129730dc => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:15.749932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc841e-9b9f-493e-848e-ac1fd5c88c22 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:23:15.855415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc841e-9b9f-493e-848e-ac1fd5c88c22 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:23:15.873214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:15.892575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d798396-ed57-43f2-8305-1c4eba65c7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:23:15.893226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d798396-ed57-43f2-8305-1c4eba65c7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:23:15.893510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d798396-ed57-43f2-8305-1c4eba65c7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:23:15.893783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d798396-ed57-43f2-8305-1c4eba65c7c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:23:15.894624 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d798396-ed57-43f2-8305-1c4eba65c7c0 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=8981b1f921ba44138b96ad4b129730dc, user.id=53af0d3c76004ff1824136fe851eac47) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:23:15.894934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d798396-ed57-43f2-8305-1c4eba65c7c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:23:15.896415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d798396-ed57-43f2-8305-1c4eba65c7c0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:23:15.902609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 599/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:23:15 2022] GET /identity/v3/role_assignments?user.id=53af0d3c76004ff1824136fe851eac47&scope.project.id=8981b1f921ba44138b96ad4b129730dc&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:23:25.728457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:23:25.899909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:11.294835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b37d9-1c37-447d-90b0-abb6f6315d4a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:11.295444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b37d9-1c37-447d-90b0-abb6f6315d4a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:11.295444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b37d9-1c37-447d-90b0-abb6f6315d4a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:11.296786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 602/1201] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:11.302582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa159988-871a-4496-8a2f-3c276acb01c6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:11.303070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa159988-871a-4496-8a2f-3c276acb01c6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:11.303175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa159988-871a-4496-8a2f-3c276acb01c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:11.322400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fa159988-871a-4496-8a2f-3c276acb01c6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:11.323634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fa159988-871a-4496-8a2f-3c276acb01c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:11.403493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 600/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:24:11.429087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b37d9-1c37-447d-90b0-abb6f6315d4a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:11.566171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b37d9-1c37-447d-90b0-abb6f6315d4a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:11.607986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8039c1-112c-4d15-a31f-9ec019134ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:11.610373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8039c1-112c-4d15-a31f-9ec019134ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:11.610373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8039c1-112c-4d15-a31f-9ec019134ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:11.610373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8039c1-112c-4d15-a31f-9ec019134ee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:11.623826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8039c1-112c-4d15-a31f-9ec019134ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:11.625454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8039c1-112c-4d15-a31f-9ec019134ee6 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-2c8039c1-112c-4d15-a31f-9ec019134ee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:11.647416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:11.648745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:11.650004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:11.650004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:11.650004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:11.650004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:11.651164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 603/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:24:11.657277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa159988-871a-4496-8a2f-3c276acb01c6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:11.768719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa159988-871a-4496-8a2f-3c276acb01c6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:11.800681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:11.808066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a20bbc-e98e-4fa2-bc8d-2a6e19f0e90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:11.808738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a20bbc-e98e-4fa2-bc8d-2a6e19f0e90f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:11.809010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a20bbc-e98e-4fa2-bc8d-2a6e19f0e90f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:11.809287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a20bbc-e98e-4fa2-bc8d-2a6e19f0e90f None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:11.810035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a20bbc-e98e-4fa2-bc8d-2a6e19f0e90f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:11.810326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a20bbc-e98e-4fa2-bc8d-2a6e19f0e90f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:11.811759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a20bbc-e98e-4fa2-bc8d-2a6e19f0e90f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:11.824764 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 601/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:11.830593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8039c1-112c-4d15-a31f-9ec019134ee6 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:11.901626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:11.964845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8039c1-112c-4d15-a31f-9ec019134ee6 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:12.008912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcbf04-e0d9-4168-b356-489982b10b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:12.009832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcbf04-e0d9-4168-b356-489982b10b3a None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:12.010069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcbf04-e0d9-4168-b356-489982b10b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:12.010471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcbf04-e0d9-4168-b356-489982b10b3a None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:12.011348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcbf04-e0d9-4168-b356-489982b10b3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:12.013167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcbf04-e0d9-4168-b356-489982b10b3a None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:12.039808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 604/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Sat Jul 2 03:24:11 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:24:14.290907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbe9df-8d5f-418e-9401-813978b16e6c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:14.291518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbe9df-8d5f-418e-9401-813978b16e6c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:14.291518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbe9df-8d5f-418e-9401-813978b16e6c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:14.293044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 602/1206] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:14.298282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969282e7-64a5-4c2b-b18f-80515ed00b1d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:14.298811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969282e7-64a5-4c2b-b18f-80515ed00b1d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:14.298901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969282e7-64a5-4c2b-b18f-80515ed00b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:14.319529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-969282e7-64a5-4c2b-b18f-80515ed00b1d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:14.319529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-969282e7-64a5-4c2b-b18f-80515ed00b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:14.414237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 605/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:24:14.439963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbe9df-8d5f-418e-9401-813978b16e6c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:14.557011 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbe9df-8d5f-418e-9401-813978b16e6c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:14.610424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37abc48-565f-47c8-a4fd-39a7d0f29537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:14.610424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37abc48-565f-47c8-a4fd-39a7d0f29537 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:14.610424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37abc48-565f-47c8-a4fd-39a7d0f29537 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:14.610424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37abc48-565f-47c8-a4fd-39a7d0f29537 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:14.622324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37abc48-565f-47c8-a4fd-39a7d0f29537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c37abc48-565f-47c8-a4fd-39a7d0f29537 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:24:14.630878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:14.631953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:14.633420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 603/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:24:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:14.638381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969282e7-64a5-4c2b-b18f-80515ed00b1d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:14.683523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:14.757660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969282e7-64a5-4c2b-b18f-80515ed00b1d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:14.802568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254bd22-fc98-4cf0-82ca-f19b0bb8ddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:14.803784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254bd22-fc98-4cf0-82ca-f19b0bb8ddd7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:14.804172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254bd22-fc98-4cf0-82ca-f19b0bb8ddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:14.804585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254bd22-fc98-4cf0-82ca-f19b0bb8ddd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:14.805732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254bd22-fc98-4cf0-82ca-f19b0bb8ddd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:14.806457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254bd22-fc98-4cf0-82ca-f19b0bb8ddd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:14.808160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254bd22-fc98-4cf0-82ca-f19b0bb8ddd7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:14.817928 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 606/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:24:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:14.823977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37abc48-565f-47c8-a4fd-39a7d0f29537 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:14.893532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:14.949411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37abc48-565f-47c8-a4fd-39a7d0f29537 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:14.984071 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674d3e3-6755-4091-b1a9-9de3940fa9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:14.984946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674d3e3-6755-4091-b1a9-9de3940fa9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:14.985228 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674d3e3-6755-4091-b1a9-9de3940fa9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:14.985799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674d3e3-6755-4091-b1a9-9de3940fa9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:14.997129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674d3e3-6755-4091-b1a9-9de3940fa9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:14.998701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674d3e3-6755-4091-b1a9-9de3940fa9e2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5674d3e3-6755-4091-b1a9-9de3940fa9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:15.017203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:15.018221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:15.019127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:15.019127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:15.019127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:15.019127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:15.019127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 604/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:24:15.024696 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254bd22-fc98-4cf0-82ca-f19b0bb8ddd7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:15.150238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:15.153677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254bd22-fc98-4cf0-82ca-f19b0bb8ddd7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:15.193773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aa081-9ceb-43da-83b0-678f1761e908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:15.194595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aa081-9ceb-43da-83b0-678f1761e908 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:15.194899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aa081-9ceb-43da-83b0-678f1761e908 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:15.195163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5aa081-9ceb-43da-83b0-678f1761e908 None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:15.195964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5aa081-9ceb-43da-83b0-678f1761e908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:15.196319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5aa081-9ceb-43da-83b0-678f1761e908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:15.197950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5aa081-9ceb-43da-83b0-678f1761e908 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:15.211467 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 607/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:15.216932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674d3e3-6755-4091-b1a9-9de3940fa9e2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:15.327233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674d3e3-6755-4091-b1a9-9de3940fa9e2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:15.329013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:15.370153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a79b15-fa07-4bb1-b16b-dc4d07a344cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:15.370948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a79b15-fa07-4bb1-b16b-dc4d07a344cd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:15.371230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a79b15-fa07-4bb1-b16b-dc4d07a344cd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:15.371535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a79b15-fa07-4bb1-b16b-dc4d07a344cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:15.380312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a79b15-fa07-4bb1-b16b-dc4d07a344cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:15.382367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a79b15-fa07-4bb1-b16b-dc4d07a344cd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-58a79b15-fa07-4bb1-b16b-dc4d07a344cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:24:15.403225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:15.404485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:15.405474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 605/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:24:15.410278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aa081-9ceb-43da-83b0-678f1761e908 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:15.540340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5aa081-9ceb-43da-83b0-678f1761e908 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:15.583441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d48a20-eaa6-439c-99d5-b4e825668345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:15.584528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d48a20-eaa6-439c-99d5-b4e825668345 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:15.584961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d48a20-eaa6-439c-99d5-b4e825668345 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:15.585496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d48a20-eaa6-439c-99d5-b4e825668345 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:15.586880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d48a20-eaa6-439c-99d5-b4e825668345 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:15.587450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d48a20-eaa6-439c-99d5-b4e825668345 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:15.589167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d48a20-eaa6-439c-99d5-b4e825668345 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:15.612068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 608/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Jul 2 03:24:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:24:15.619265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a79b15-fa07-4bb1-b16b-dc4d07a344cd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:15.747541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a79b15-fa07-4bb1-b16b-dc4d07a344cd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:15.788050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd04ec-841e-4d72-919e-311d8b183b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:15.788752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd04ec-841e-4d72-919e-311d8b183b1f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:15.789014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd04ec-841e-4d72-919e-311d8b183b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:15.789293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd04ec-841e-4d72-919e-311d8b183b1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:15.825525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:15.869194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd04ec-841e-4d72-919e-311d8b183b1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:15.870736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd04ec-841e-4d72-919e-311d8b183b1f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:15.979724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 606/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:24:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:24:15.986406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d48a20-eaa6-439c-99d5-b4e825668345 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:16.007819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:16.113162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d48a20-eaa6-439c-99d5-b4e825668345 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:16.153658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3eb48-1fdd-42fc-b928-042455c179d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:16.154565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3eb48-1fdd-42fc-b928-042455c179d0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:16.154845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3eb48-1fdd-42fc-b928-042455c179d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:16.155116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3eb48-1fdd-42fc-b928-042455c179d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:16.170143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3eb48-1fdd-42fc-b928-042455c179d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:16.172189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3eb48-1fdd-42fc-b928-042455c179d0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b1c3eb48-1fdd-42fc-b928-042455c179d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:16.193981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:16.195130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:16.196269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:16.196269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:16.196269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:16.196269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:16.199463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 609/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:16.201577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd04ec-841e-4d72-919e-311d8b183b1f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:16.274139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:16.317700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd04ec-841e-4d72-919e-311d8b183b1f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:16.358249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2d4cd-8c95-4591-add5-72b0504c9477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:16.359010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2d4cd-8c95-4591-add5-72b0504c9477 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:16.359309 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2d4cd-8c95-4591-add5-72b0504c9477 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:16.359595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2d4cd-8c95-4591-add5-72b0504c9477 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:16.361893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2d4cd-8c95-4591-add5-72b0504c9477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:16.361893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2d4cd-8c95-4591-add5-72b0504c9477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:16.362542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2d4cd-8c95-4591-add5-72b0504c9477 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:16.377308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 607/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:24:16.382846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3eb48-1fdd-42fc-b928-042455c179d0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:16.461375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:16.495120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3eb48-1fdd-42fc-b928-042455c179d0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:16.533754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c263b-5008-42b3-a525-d1c92786d909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:16.534796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c263b-5008-42b3-a525-d1c92786d909 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:16.535265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c263b-5008-42b3-a525-d1c92786d909 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:16.535758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c263b-5008-42b3-a525-d1c92786d909 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:16.543004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c263b-5008-42b3-a525-d1c92786d909 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:16.544758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c263b-5008-42b3-a525-d1c92786d909 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-464c263b-5008-42b3-a525-d1c92786d909 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:24:16.558997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:16.560225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:16.561351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:24:16.561351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:24:16.561351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:24:16.561351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:24:16.561351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:24:16.561351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:24:16.561351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:16.561351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:16.567038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 610/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:24:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:16.568437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2d4cd-8c95-4591-add5-72b0504c9477 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:16.686679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2d4cd-8c95-4591-add5-72b0504c9477 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:16.709798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:16.721494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4d292-82b2-406f-9e7f-72d867c91ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:16.722319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4d292-82b2-406f-9e7f-72d867c91ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:16.722531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4d292-82b2-406f-9e7f-72d867c91ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:16.722808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4d292-82b2-406f-9e7f-72d867c91ce0 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:16.723789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4d292-82b2-406f-9e7f-72d867c91ce0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:16.724104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4d292-82b2-406f-9e7f-72d867c91ce0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:16.725692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4d292-82b2-406f-9e7f-72d867c91ce0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:16.739862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 608/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:24:16.746226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c263b-5008-42b3-a525-d1c92786d909 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:16.860223 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c263b-5008-42b3-a525-d1c92786d909 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:16.909782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:16.922500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fce9f-bc7a-4a66-9cf2-a18f06198685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:16.923374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fce9f-bc7a-4a66-9cf2-a18f06198685 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:16.924024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fce9f-bc7a-4a66-9cf2-a18f06198685 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:16.924448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fce9f-bc7a-4a66-9cf2-a18f06198685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:16.925757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fce9f-bc7a-4a66-9cf2-a18f06198685 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=6ee7acbd4c5545c1885faffb71f6306b) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:16.926118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fce9f-bc7a-4a66-9cf2-a18f06198685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:16.929020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fce9f-bc7a-4a66-9cf2-a18f06198685 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:16.939521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 611/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:24:16 2022] GET /identity/v3/role_assignments?user.id=6ee7acbd4c5545c1885faffb71f6306b&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:19.196195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2b527-34b5-4829-a550-ba0f2958ad89 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:19.196770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2b527-34b5-4829-a550-ba0f2958ad89 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:19.196900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb2b527-34b5-4829-a550-ba0f2958ad89 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:19.198601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 609/1220] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:19.204241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be65c3-0eb2-4ecd-bc44-e18631f3df80 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:19.204668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be65c3-0eb2-4ecd-bc44-e18631f3df80 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:19.204789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be65c3-0eb2-4ecd-bc44-e18631f3df80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:19.231935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-21be65c3-0eb2-4ecd-bc44-e18631f3df80 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:19.234117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-21be65c3-0eb2-4ecd-bc44-e18631f3df80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:19.322181 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 612/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:24:19.345385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2b527-34b5-4829-a550-ba0f2958ad89 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:19.458101 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb2b527-34b5-4829-a550-ba0f2958ad89 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:19.494777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdaff72-06e5-4333-8b03-72149f1e36f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:19.495609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaff72-06e5-4333-8b03-72149f1e36f9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:19.495892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaff72-06e5-4333-8b03-72149f1e36f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:19.496166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaff72-06e5-4333-8b03-72149f1e36f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:19.506624 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdaff72-06e5-4333-8b03-72149f1e36f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3fdaff72-06e5-4333-8b03-72149f1e36f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:24:19.516330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:19.517487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:19.518423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 610/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:19.524789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be65c3-0eb2-4ecd-bc44-e18631f3df80 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:19.635856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be65c3-0eb2-4ecd-bc44-e18631f3df80 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:19.661342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:19.674216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f5a4b-06e6-49a9-883e-7f281d608749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:19.674884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f5a4b-06e6-49a9-883e-7f281d608749 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:19.675236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f5a4b-06e6-49a9-883e-7f281d608749 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:19.675545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f5a4b-06e6-49a9-883e-7f281d608749 None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:19.676321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f5a4b-06e6-49a9-883e-7f281d608749 None admin] RBAC: Adding query filter params (name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:19.676630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f5a4b-06e6-49a9-883e-7f281d608749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:19.678011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f5a4b-06e6-49a9-883e-7f281d608749 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:19.686607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 613/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:24:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:19.692682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdaff72-06e5-4333-8b03-72149f1e36f9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:19.738073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:19.795905 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdaff72-06e5-4333-8b03-72149f1e36f9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:19.830355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc2b01-8fb4-4f13-b307-496c681b2102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:19.831160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc2b01-8fb4-4f13-b307-496c681b2102 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:19.831469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc2b01-8fb4-4f13-b307-496c681b2102 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:19.831741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc2b01-8fb4-4f13-b307-496c681b2102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:19.842294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc2b01-8fb4-4f13-b307-496c681b2102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:19.843802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc2b01-8fb4-4f13-b307-496c681b2102 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-01dc2b01-8fb4-4f13-b307-496c681b2102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:19.860929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:19.861956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:19.863012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:19.863012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:19.863012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:19.863012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:19.863012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 611/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:19.868334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f5a4b-06e6-49a9-883e-7f281d608749 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:19.983086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f5a4b-06e6-49a9-883e-7f281d608749 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:20.018908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fa8da-b1c9-41f1-8208-57855e9bbfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:20.019658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fa8da-b1c9-41f1-8208-57855e9bbfba None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:20.020142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fa8da-b1c9-41f1-8208-57855e9bbfba None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:20.020559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fa8da-b1c9-41f1-8208-57855e9bbfba None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:20.021407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fa8da-b1c9-41f1-8208-57855e9bbfba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:20.021881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fa8da-b1c9-41f1-8208-57855e9bbfba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:20.023390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fa8da-b1c9-41f1-8208-57855e9bbfba None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:20.036019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 614/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:24:20.042064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc2b01-8fb4-4f13-b307-496c681b2102 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:20.143975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:20.151279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc2b01-8fb4-4f13-b307-496c681b2102 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:20.191338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fcf51-2735-421c-89d1-ae6756c75075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:20.192336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fcf51-2735-421c-89d1-ae6756c75075 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:20.192768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fcf51-2735-421c-89d1-ae6756c75075 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:20.193072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3fcf51-2735-421c-89d1-ae6756c75075 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:20.202873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fcf51-2735-421c-89d1-ae6756c75075 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:20.204464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3fcf51-2735-421c-89d1-ae6756c75075 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1f3fcf51-2735-421c-89d1-ae6756c75075 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:24:20.220207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:20.221206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:20.222143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 612/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:20.227591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fa8da-b1c9-41f1-8208-57855e9bbfba None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:20.277303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:20.332746 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fa8da-b1c9-41f1-8208-57855e9bbfba None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:20.368704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce090d76-cf91-4cbe-a2b6-287585c44811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:20.369357 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090d76-cf91-4cbe-a2b6-287585c44811 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:20.369638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090d76-cf91-4cbe-a2b6-287585c44811 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:20.369912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce090d76-cf91-4cbe-a2b6-287585c44811 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:20.370756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce090d76-cf91-4cbe-a2b6-287585c44811 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:20.371069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce090d76-cf91-4cbe-a2b6-287585c44811 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:20.372549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce090d76-cf91-4cbe-a2b6-287585c44811 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:20.391688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 615/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Jul 2 03:24:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:20.397196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fcf51-2735-421c-89d1-ae6756c75075 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:20.508228 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3fcf51-2735-421c-89d1-ae6756c75075 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:20.547134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04002d09-f151-49a1-868b-5b37b187723c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:20.547846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04002d09-f151-49a1-868b-5b37b187723c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:20.548125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04002d09-f151-49a1-868b-5b37b187723c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:20.548420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04002d09-f151-49a1-868b-5b37b187723c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:20.560351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04002d09-f151-49a1-868b-5b37b187723c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:20.562136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04002d09-f151-49a1-868b-5b37b187723c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-04002d09-f151-49a1-868b-5b37b187723c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:20.582170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:20.584008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:20.585108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:20.585108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:20.585108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:20.585108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:20.585108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 613/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:20.589855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce090d76-cf91-4cbe-a2b6-287585c44811 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:20.709575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce090d76-cf91-4cbe-a2b6-287585c44811 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:20.712309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:20.751739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6d533-0d33-4548-bdf1-2b8a0756974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:20.752583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6d533-0d33-4548-bdf1-2b8a0756974b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:20.752917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6d533-0d33-4548-bdf1-2b8a0756974b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:20.753438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6d533-0d33-4548-bdf1-2b8a0756974b None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:20.754270 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6d533-0d33-4548-bdf1-2b8a0756974b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:20.754568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6d533-0d33-4548-bdf1-2b8a0756974b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:20.756373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6d533-0d33-4548-bdf1-2b8a0756974b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:20.776262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 616/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:20.778038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04002d09-f151-49a1-868b-5b37b187723c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:20.790104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:20.891583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04002d09-f151-49a1-868b-5b37b187723c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:20.934938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d7f08-d77e-4d4c-a84a-911c958685fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:20.935781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d7f08-d77e-4d4c-a84a-911c958685fb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:20.936058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d7f08-d77e-4d4c-a84a-911c958685fb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:20.936325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d7f08-d77e-4d4c-a84a-911c958685fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:20.944854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d7f08-d77e-4d4c-a84a-911c958685fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:20.946605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d7f08-d77e-4d4c-a84a-911c958685fb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e41d7f08-d77e-4d4c-a84a-911c958685fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:24:20.964694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:20.966063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:20.967225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:24:20.967225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:24:20.967225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:24:20.967225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:24:20.967225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:24:20.967225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:24:20.967225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:20.967225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:20.967225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 614/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:24:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:20.973484 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6d533-0d33-4548-bdf1-2b8a0756974b None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:21.089792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6d533-0d33-4548-bdf1-2b8a0756974b None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:21.130053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fb1af-18ee-457a-a43e-23f7a9006550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:21.130744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fb1af-18ee-457a-a43e-23f7a9006550 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:21.131022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fb1af-18ee-457a-a43e-23f7a9006550 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:21.131323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fb1af-18ee-457a-a43e-23f7a9006550 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:21.132127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fb1af-18ee-457a-a43e-23f7a9006550 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:21.132442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fb1af-18ee-457a-a43e-23f7a9006550 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:21.133873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fb1af-18ee-457a-a43e-23f7a9006550 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:21.148370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 617/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:24:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:21.154444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d7f08-d77e-4d4c-a84a-911c958685fb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:21.228236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:21.268902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d7f08-d77e-4d4c-a84a-911c958685fb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:21.310838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3e0fe-51a2-4bcb-bc8e-d1995e9549b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:21.311934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3e0fe-51a2-4bcb-bc8e-d1995e9549b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:21.312318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3e0fe-51a2-4bcb-bc8e-d1995e9549b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:21.312709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3e0fe-51a2-4bcb-bc8e-d1995e9549b1 None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/6ee7acbd4c5545c1885faffb71f6306b/roles/4a54555205fa406897a3abc7871ee1aa` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:21.328404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3e0fe-51a2-4bcb-bc8e-d1995e9549b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=6ee7acbd4c5545c1885faffb71f6306b, role_id=4a54555205fa406897a3abc7871ee1aa)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:21.329900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3e0fe-51a2-4bcb-bc8e-d1995e9549b1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:21.343372 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 615/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:24:21 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/6ee7acbd4c5545c1885faffb71f6306b/roles/4a54555205fa406897a3abc7871ee1aa => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:24:23.479188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514bb9-35fe-4a27-93c5-77456250e206 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:23.479827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514bb9-35fe-4a27-93c5-77456250e206 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:23.479827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88514bb9-35fe-4a27-93c5-77456250e206 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:23.481140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 618/1233] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:23.486456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cd554-f158-4041-96f1-5cd6984b6bb3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:23.486907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cd554-f158-4041-96f1-5cd6984b6bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:23.487033 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503cd554-f158-4041-96f1-5cd6984b6bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:23.504710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-503cd554-f158-4041-96f1-5cd6984b6bb3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:23.505998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-503cd554-f158-4041-96f1-5cd6984b6bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:23.577005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 616/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:24:23.601814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88514bb9-35fe-4a27-93c5-77456250e206 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:23.629548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:23.713814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88514bb9-35fe-4a27-93c5-77456250e206 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:23.747921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402891f-2898-4617-baba-6ea2a2e9a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:23.748638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402891f-2898-4617-baba-6ea2a2e9a20a None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:23.748924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402891f-2898-4617-baba-6ea2a2e9a20a None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:23.749209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402891f-2898-4617-baba-6ea2a2e9a20a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:23.759138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402891f-2898-4617-baba-6ea2a2e9a20a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-d402891f-2898-4617-baba-6ea2a2e9a20a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:24:23.766829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:23.767996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:23.768940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 619/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Sat Jul 2 03:24:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 02 03:24:23.774542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503cd554-f158-4041-96f1-5cd6984b6bb3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:23.823871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:23.890392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503cd554-f158-4041-96f1-5cd6984b6bb3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:23.930425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93a265-d71e-4f82-a49f-e96596acbb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:23.931100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93a265-d71e-4f82-a49f-e96596acbb7a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:23.931724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93a265-d71e-4f82-a49f-e96596acbb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:23.932000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93a265-d71e-4f82-a49f-e96596acbb7a None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:23.932743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93a265-d71e-4f82-a49f-e96596acbb7a None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:23.933049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93a265-d71e-4f82-a49f-e96596acbb7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:23.934431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93a265-d71e-4f82-a49f-e96596acbb7a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:23.942924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 617/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Sat Jul 2 03:24:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:23.948835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402891f-2898-4617-baba-6ea2a2e9a20a None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:24.040052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:24.058593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402891f-2898-4617-baba-6ea2a2e9a20a None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:24.097912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333687c-8d51-4e59-bc4d-fe8b27ff61b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:24.098912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333687c-8d51-4e59-bc4d-fe8b27ff61b5 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:24.099330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333687c-8d51-4e59-bc4d-fe8b27ff61b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:24.100093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333687c-8d51-4e59-bc4d-fe8b27ff61b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:24.113179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333687c-8d51-4e59-bc4d-fe8b27ff61b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:24.115102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333687c-8d51-4e59-bc4d-fe8b27ff61b5 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-b333687c-8d51-4e59-bc4d-fe8b27ff61b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:24.134914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:24.135919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:24.136832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:24.136832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:24.136832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:24.136832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:24.136832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 620/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:24.142562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93a265-d71e-4f82-a49f-e96596acbb7a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:24.257891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93a265-d71e-4f82-a49f-e96596acbb7a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:24.267595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:24.294557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90d95e-64be-405f-b72e-c6b7afebe02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:24.295150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90d95e-64be-405f-b72e-c6b7afebe02d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:24.295468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90d95e-64be-405f-b72e-c6b7afebe02d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:24.295742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90d95e-64be-405f-b72e-c6b7afebe02d None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:24.296465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90d95e-64be-405f-b72e-c6b7afebe02d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:24.296802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90d95e-64be-405f-b72e-c6b7afebe02d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:24.298146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90d95e-64be-405f-b72e-c6b7afebe02d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:24.310405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 618/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:24:24.316379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333687c-8d51-4e59-bc4d-fe8b27ff61b5 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:24.440479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333687c-8d51-4e59-bc4d-fe8b27ff61b5 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:24.476600 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651b270-8dd8-49ee-ad71-fe6706bf2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:24.477544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651b270-8dd8-49ee-ad71-fe6706bf2e86 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:24.477829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651b270-8dd8-49ee-ad71-fe6706bf2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:24.478107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651b270-8dd8-49ee-ad71-fe6706bf2e86 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:24.481499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:24.487487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651b270-8dd8-49ee-ad71-fe6706bf2e86 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:24.489367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651b270-8dd8-49ee-ad71-fe6706bf2e86 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-a651b270-8dd8-49ee-ad71-fe6706bf2e86 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:24:24.505733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:24.506791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:24.507824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 621/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:24.513276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90d95e-64be-405f-b72e-c6b7afebe02d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:24.626954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90d95e-64be-405f-b72e-c6b7afebe02d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:24.668728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab07a10-87e7-4114-90f9-7d2b44393069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:24.669771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab07a10-87e7-4114-90f9-7d2b44393069 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:24.670146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab07a10-87e7-4114-90f9-7d2b44393069 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:24.670534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab07a10-87e7-4114-90f9-7d2b44393069 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:24.671744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab07a10-87e7-4114-90f9-7d2b44393069 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:24.672314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab07a10-87e7-4114-90f9-7d2b44393069 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:24.674864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab07a10-87e7-4114-90f9-7d2b44393069 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:24.705118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 619/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Jul 2 03:24:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:24.711134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651b270-8dd8-49ee-ad71-fe6706bf2e86 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:24.828662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651b270-8dd8-49ee-ad71-fe6706bf2e86 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:24.870397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a56da4-3b21-4336-98be-6a2a946967d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:24.871082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a56da4-3b21-4336-98be-6a2a946967d7 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:24.871416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a56da4-3b21-4336-98be-6a2a946967d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:24.871745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a56da4-3b21-4336-98be-6a2a946967d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:24.947726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:24.955356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a56da4-3b21-4336-98be-6a2a946967d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:24.957013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a56da4-3b21-4336-98be-6a2a946967d7 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:25.066917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 622/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:24:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:24:25.075504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab07a10-87e7-4114-90f9-7d2b44393069 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:25.111672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:25.201484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab07a10-87e7-4114-90f9-7d2b44393069 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:25.240819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d7916-d76f-4b18-96e8-5e1c711a05c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:25.241617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d7916-d76f-4b18-96e8-5e1c711a05c6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:25.241919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d7916-d76f-4b18-96e8-5e1c711a05c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:25.242172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d7916-d76f-4b18-96e8-5e1c711a05c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:25.252655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d7916-d76f-4b18-96e8-5e1c711a05c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:25.254107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d7916-d76f-4b18-96e8-5e1c711a05c6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f22d7916-d76f-4b18-96e8-5e1c711a05c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:25.273421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:25.274541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:25.275617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:25.275617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:25.275617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:25.275617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:25.275617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 620/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:25.280702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a56da4-3b21-4336-98be-6a2a946967d7 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:25.388305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:25.395407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a56da4-3b21-4336-98be-6a2a946967d7 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:25.434928 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f844a-c8a3-4de9-92f7-2400fbbd6abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:25.435629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f844a-c8a3-4de9-92f7-2400fbbd6abb None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:25.435909 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f844a-c8a3-4de9-92f7-2400fbbd6abb None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:25.436185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f844a-c8a3-4de9-92f7-2400fbbd6abb None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:25.436901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f844a-c8a3-4de9-92f7-2400fbbd6abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:25.437203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f844a-c8a3-4de9-92f7-2400fbbd6abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:25.438594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f844a-c8a3-4de9-92f7-2400fbbd6abb None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:25.451643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 623/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:25.457452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d7916-d76f-4b18-96e8-5e1c711a05c6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:25.540504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:25.564457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d7916-d76f-4b18-96e8-5e1c711a05c6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:25.600898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b89db6-ce81-40be-932c-7c8c46776df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:25.601707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b89db6-ce81-40be-932c-7c8c46776df8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:25.601994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b89db6-ce81-40be-932c-7c8c46776df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:25.602268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b89db6-ce81-40be-932c-7c8c46776df8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:25.609161 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b89db6-ce81-40be-932c-7c8c46776df8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:25.610622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b89db6-ce81-40be-932c-7c8c46776df8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-18b89db6-ce81-40be-932c-7c8c46776df8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:24:25.624787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:25.626147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:25.627408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:24:25.627408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:24:25.627408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:24:25.627408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:24:25.627408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:24:25.627408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:24:25.627408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:25.627408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:25.631199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 621/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:25.632215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f844a-c8a3-4de9-92f7-2400fbbd6abb None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:25.741416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f844a-c8a3-4de9-92f7-2400fbbd6abb None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:25.781348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8326f4-a18d-4199-97c2-21ac2b1b7910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:25.782059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8326f4-a18d-4199-97c2-21ac2b1b7910 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:25.782370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8326f4-a18d-4199-97c2-21ac2b1b7910 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:25.782740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8326f4-a18d-4199-97c2-21ac2b1b7910 None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:25.783835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8326f4-a18d-4199-97c2-21ac2b1b7910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:25.784250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8326f4-a18d-4199-97c2-21ac2b1b7910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:25.786061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8326f4-a18d-4199-97c2-21ac2b1b7910 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:25.799893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 624/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:24:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:25.807153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b89db6-ce81-40be-932c-7c8c46776df8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:25.934866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b89db6-ce81-40be-932c-7c8c46776df8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:25.972128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654ee89-9a9a-4f01-b12c-629ff5e6d1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:25.972772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654ee89-9a9a-4f01-b12c-629ff5e6d1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:25.973018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654ee89-9a9a-4f01-b12c-629ff5e6d1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:25.973316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654ee89-9a9a-4f01-b12c-629ff5e6d1e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:25.974372 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654ee89-9a9a-4f01-b12c-629ff5e6d1e2 None admin] RBAC: Adding query filter params (role.id=4a54555205fa406897a3abc7871ee1aa, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=6ee7acbd4c5545c1885faffb71f6306b) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:25.974717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654ee89-9a9a-4f01-b12c-629ff5e6d1e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:25.976239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654ee89-9a9a-4f01-b12c-629ff5e6d1e2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:25.982636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 622/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:24:25 2022] GET /identity/v3/role_assignments?user.id=6ee7acbd4c5545c1885faffb71f6306b&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=4a54555205fa406897a3abc7871ee1aa => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:28.376601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48395b-5190-45aa-8f39-47e4e5cefddd None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:28.377414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48395b-5190-45aa-8f39-47e4e5cefddd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:28.377414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48395b-5190-45aa-8f39-47e4e5cefddd None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:28.378642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 625/1247] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:28.386527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2d33-fba2-4dc6-802a-a40588698fbd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:28.387034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2d33-fba2-4dc6-802a-a40588698fbd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:28.387216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c2d33-fba2-4dc6-802a-a40588698fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:28.393930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:28.409993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-148c2d33-fba2-4dc6-802a-a40588698fbd None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:28.411541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-148c2d33-fba2-4dc6-802a-a40588698fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:28.493187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 623/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:24:28.517623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48395b-5190-45aa-8f39-47e4e5cefddd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:28.538616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:28.641451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48395b-5190-45aa-8f39-47e4e5cefddd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:28.681546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae3b56-495d-4dff-b12d-1d76b7128989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:28.682360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae3b56-495d-4dff-b12d-1d76b7128989 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:28.682656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae3b56-495d-4dff-b12d-1d76b7128989 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:28.682980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae3b56-495d-4dff-b12d-1d76b7128989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:28.695348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae3b56-495d-4dff-b12d-1d76b7128989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-1dae3b56-495d-4dff-b12d-1d76b7128989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:24:28.703406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:28.704622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:28.711250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 626/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:24:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:28.713218 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c2d33-fba2-4dc6-802a-a40588698fbd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:28.830606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c2d33-fba2-4dc6-802a-a40588698fbd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:28.866318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:28.872202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db6b9a-ca1d-4401-84f0-096b5e718189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:28.873028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db6b9a-ca1d-4401-84f0-096b5e718189 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:28.873405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db6b9a-ca1d-4401-84f0-096b5e718189 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:28.873723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db6b9a-ca1d-4401-84f0-096b5e718189 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:28.874456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db6b9a-ca1d-4401-84f0-096b5e718189 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:28.874761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db6b9a-ca1d-4401-84f0-096b5e718189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:28.876259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db6b9a-ca1d-4401-84f0-096b5e718189 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:28.885830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 624/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:28.893159 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae3b56-495d-4dff-b12d-1d76b7128989 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:28.980251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:29.007891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae3b56-495d-4dff-b12d-1d76b7128989 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:29.046021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188031e3-878f-4960-ae3e-297450aaa8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:29.046959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188031e3-878f-4960-ae3e-297450aaa8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:29.047279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188031e3-878f-4960-ae3e-297450aaa8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:29.047582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188031e3-878f-4960-ae3e-297450aaa8a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:29.059558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188031e3-878f-4960-ae3e-297450aaa8a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:29.061119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188031e3-878f-4960-ae3e-297450aaa8a0 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-188031e3-878f-4960-ae3e-297450aaa8a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:29.078882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:29.080012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:29.081154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:29.081154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:29.081154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:29.081154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:29.081154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 627/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:29.087147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db6b9a-ca1d-4401-84f0-096b5e718189 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:29.206977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db6b9a-ca1d-4401-84f0-096b5e718189 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:29.242434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc81e8-ef59-4fc3-93b5-c22e423d48dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:29.243130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc81e8-ef59-4fc3-93b5-c22e423d48dd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:29.243445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc81e8-ef59-4fc3-93b5-c22e423d48dd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:29.243720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc81e8-ef59-4fc3-93b5-c22e423d48dd None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:29.244469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc81e8-ef59-4fc3-93b5-c22e423d48dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:29.244775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc81e8-ef59-4fc3-93b5-c22e423d48dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:29.246173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc81e8-ef59-4fc3-93b5-c22e423d48dd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:29.263044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 625/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:29.269296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188031e3-878f-4960-ae3e-297450aaa8a0 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:29.378590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188031e3-878f-4960-ae3e-297450aaa8a0 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:29.407447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:29.417576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1add8c-5c57-4a2f-8d0a-27f730cc21ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:29.418392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1add8c-5c57-4a2f-8d0a-27f730cc21ac None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:29.418674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1add8c-5c57-4a2f-8d0a-27f730cc21ac None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:29.418946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1add8c-5c57-4a2f-8d0a-27f730cc21ac None admin] PATH_INFO: `/v3/users/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:29.428838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1add8c-5c57-4a2f-8d0a-27f730cc21ac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:29.430714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1add8c-5c57-4a2f-8d0a-27f730cc21ac None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-6e1add8c-5c57-4a2f-8d0a-27f730cc21ac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:24:29.448087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:29.450105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:29.452259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 628/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:29.455606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc81e8-ef59-4fc3-93b5-c22e423d48dd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:29.521053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:29.572045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc81e8-ef59-4fc3-93b5-c22e423d48dd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:29.611440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909716b-be91-4828-860d-48b082d3647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:29.612125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909716b-be91-4828-860d-48b082d3647d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:29.612404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909716b-be91-4828-860d-48b082d3647d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:29.612676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909716b-be91-4828-860d-48b082d3647d None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:29.613504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909716b-be91-4828-860d-48b082d3647d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:29.613893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909716b-be91-4828-860d-48b082d3647d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:29.615825 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909716b-be91-4828-860d-48b082d3647d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:29.637041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 626/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Jul 2 03:24:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:29.643812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1add8c-5c57-4a2f-8d0a-27f730cc21ac None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:29.762127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1add8c-5c57-4a2f-8d0a-27f730cc21ac None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:29.801230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f3220-14f3-48b5-93dd-da734313c57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:29.802136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3220-14f3-48b5-93dd-da734313c57f None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:29.802416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3220-14f3-48b5-93dd-da734313c57f None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:29.802725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3220-14f3-48b5-93dd-da734313c57f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:29.861379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:29.879639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f3220-14f3-48b5-93dd-da734313c57f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:29.881564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f3220-14f3-48b5-93dd-da734313c57f None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:29.998146 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 629/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:24:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:24:30.004308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909716b-be91-4828-860d-48b082d3647d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:30.116813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909716b-be91-4828-860d-48b082d3647d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:30.131808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:30.159460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c64f8-0749-4e2c-947e-c584c16c2e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:30.163706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c64f8-0749-4e2c-947e-c584c16c2e9d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:30.163989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c64f8-0749-4e2c-947e-c584c16c2e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:30.164258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c64f8-0749-4e2c-947e-c584c16c2e9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:30.176911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c64f8-0749-4e2c-947e-c584c16c2e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:30.178555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c64f8-0749-4e2c-947e-c584c16c2e9d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b73c64f8-0749-4e2c-947e-c584c16c2e9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:30.197898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:30.199090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:30.200110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:30.200110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:30.200110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:30.200110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:30.200110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 627/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:30.205009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f3220-14f3-48b5-93dd-da734313c57f None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:30.300821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:30.314812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f3220-14f3-48b5-93dd-da734313c57f None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:30.363386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a734618-d09c-46f7-bf3f-18cf1404651e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:30.364311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a734618-d09c-46f7-bf3f-18cf1404651e None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:30.364715 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a734618-d09c-46f7-bf3f-18cf1404651e None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:30.364995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a734618-d09c-46f7-bf3f-18cf1404651e None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:30.365796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a734618-d09c-46f7-bf3f-18cf1404651e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:30.366105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a734618-d09c-46f7-bf3f-18cf1404651e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:30.367855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a734618-d09c-46f7-bf3f-18cf1404651e None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:30.381616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 630/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:30.387764 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c64f8-0749-4e2c-947e-c584c16c2e9d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:30.504627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c64f8-0749-4e2c-947e-c584c16c2e9d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:30.541093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649ed61-5a3b-43e7-8f20-77cc5ad84ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:30.541899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649ed61-5a3b-43e7-8f20-77cc5ad84ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:30.542178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649ed61-5a3b-43e7-8f20-77cc5ad84ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:30.542449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649ed61-5a3b-43e7-8f20-77cc5ad84ea5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:30.549231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649ed61-5a3b-43e7-8f20-77cc5ad84ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:30.550675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649ed61-5a3b-43e7-8f20-77cc5ad84ea5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9649ed61-5a3b-43e7-8f20-77cc5ad84ea5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:24:30.568971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:30.570093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:30.571057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:24:30.571057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:24:30.571057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:24:30.571057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:24:30.571057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:24:30.571057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:24:30.571057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:30.571057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:30.571057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 628/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:24:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:30.576200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a734618-d09c-46f7-bf3f-18cf1404651e None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:30.685441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a734618-d09c-46f7-bf3f-18cf1404651e None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:30.722058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91be70e-c08b-42d5-8f63-607fa49c357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:30.722724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91be70e-c08b-42d5-8f63-607fa49c357b None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:30.723001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91be70e-c08b-42d5-8f63-607fa49c357b None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:30.723321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91be70e-c08b-42d5-8f63-607fa49c357b None admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:30.724105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91be70e-c08b-42d5-8f63-607fa49c357b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:30.724404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91be70e-c08b-42d5-8f63-607fa49c357b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:30.725841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91be70e-c08b-42d5-8f63-607fa49c357b None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:30.738506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 631/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:30.744985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649ed61-5a3b-43e7-8f20-77cc5ad84ea5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:30.774833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:30.852820 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649ed61-5a3b-43e7-8f20-77cc5ad84ea5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:30.895468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf7754-3fb4-41fc-aeb5-6d91492dc454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:30.896525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf7754-3fb4-41fc-aeb5-6d91492dc454 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:30.896919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf7754-3fb4-41fc-aeb5-6d91492dc454 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:30.897418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf7754-3fb4-41fc-aeb5-6d91492dc454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:30.898329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf7754-3fb4-41fc-aeb5-6d91492dc454 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=6ee7acbd4c5545c1885faffb71f6306b) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:30.898909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf7754-3fb4-41fc-aeb5-6d91492dc454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:30.900606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf7754-3fb4-41fc-aeb5-6d91492dc454 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:30.907416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 629/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:24:30 2022] GET /identity/v3/role_assignments?user.id=6ee7acbd4c5545c1885faffb71f6306b&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:32.929328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663ae22-6d56-4280-b78f-238d7eed63b0 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:32.929862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663ae22-6d56-4280-b78f-238d7eed63b0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:32.929862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663ae22-6d56-4280-b78f-238d7eed63b0 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:32.931626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 632/1261] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:32.937104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ddbb-b0df-4223-9a23-ea59b777d046 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:32.937548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ddbb-b0df-4223-9a23-ea59b777d046 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:32.937662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ddbb-b0df-4223-9a23-ea59b777d046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:32.955462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1cd6ddbb-b0df-4223-9a23-ea59b777d046 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:32.956623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1cd6ddbb-b0df-4223-9a23-ea59b777d046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:33.028524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 630/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:24:33.061628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663ae22-6d56-4280-b78f-238d7eed63b0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:33.073335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:33.185805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663ae22-6d56-4280-b78f-238d7eed63b0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:33.220338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33694f5b-2fe3-4ef7-8705-185495e59785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:33.221107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33694f5b-2fe3-4ef7-8705-185495e59785 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:33.221388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33694f5b-2fe3-4ef7-8705-185495e59785 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:33.221620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33694f5b-2fe3-4ef7-8705-185495e59785 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:33.233766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33694f5b-2fe3-4ef7-8705-185495e59785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-33694f5b-2fe3-4ef7-8705-185495e59785 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:24:33.241363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:33.242419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:33.243392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 633/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:24:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:33.249639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6ddbb-b0df-4223-9a23-ea59b777d046 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:33.270580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:33.356937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6ddbb-b0df-4223-9a23-ea59b777d046 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:33.394725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7ec4f-8fd1-4f3c-9dcc-5eb1e926b0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:33.395497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7ec4f-8fd1-4f3c-9dcc-5eb1e926b0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:33.395877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7ec4f-8fd1-4f3c-9dcc-5eb1e926b0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:33.396149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7ec4f-8fd1-4f3c-9dcc-5eb1e926b0e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:33.396891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7ec4f-8fd1-4f3c-9dcc-5eb1e926b0e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:33.397195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7ec4f-8fd1-4f3c-9dcc-5eb1e926b0e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:33.398560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7ec4f-8fd1-4f3c-9dcc-5eb1e926b0e6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:33.406848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 631/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:24:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:33.413470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33694f5b-2fe3-4ef7-8705-185495e59785 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:33.494574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:33.535011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33694f5b-2fe3-4ef7-8705-185495e59785 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:33.573113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489b1d-7919-4763-89d9-28ef9626655d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:33.573911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489b1d-7919-4763-89d9-28ef9626655d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:33.574197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489b1d-7919-4763-89d9-28ef9626655d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:33.574472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489b1d-7919-4763-89d9-28ef9626655d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:33.586394 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84489b1d-7919-4763-89d9-28ef9626655d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:33.587898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84489b1d-7919-4763-89d9-28ef9626655d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-84489b1d-7919-4763-89d9-28ef9626655d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:33.605200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:33.606252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:33.612064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:33.612064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:33.612064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:33.612064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:33.612064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 634/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:33.612392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7ec4f-8fd1-4f3c-9dcc-5eb1e926b0e6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:33.704415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:33.724927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7ec4f-8fd1-4f3c-9dcc-5eb1e926b0e6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:33.760193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f652f-6dac-4a56-9109-0717e9818880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:33.760777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f652f-6dac-4a56-9109-0717e9818880 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:33.761063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f652f-6dac-4a56-9109-0717e9818880 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:33.761336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f652f-6dac-4a56-9109-0717e9818880 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:33.762039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f652f-6dac-4a56-9109-0717e9818880 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:33.762347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f652f-6dac-4a56-9109-0717e9818880 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:33.763754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f652f-6dac-4a56-9109-0717e9818880 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:33.775761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 632/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:33.784852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489b1d-7919-4763-89d9-28ef9626655d None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:33.909993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489b1d-7919-4763-89d9-28ef9626655d None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:33.932506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:33.947496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52d605-587f-481f-a13f-35e0c9325c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:33.948299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52d605-587f-481f-a13f-35e0c9325c95 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:33.948577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52d605-587f-481f-a13f-35e0c9325c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:33.948847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52d605-587f-481f-a13f-35e0c9325c95 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:33.956396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52d605-587f-481f-a13f-35e0c9325c95 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:33.957970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52d605-587f-481f-a13f-35e0c9325c95 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-8f52d605-587f-481f-a13f-35e0c9325c95 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:24:33.972408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:33.973657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:33.975034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 635/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:33.982986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f652f-6dac-4a56-9109-0717e9818880 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:34.095278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f652f-6dac-4a56-9109-0717e9818880 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:34.139927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c865f-0b28-47c8-b7ed-3209992fe00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:34.140867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c865f-0b28-47c8-b7ed-3209992fe00c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:34.141216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c865f-0b28-47c8-b7ed-3209992fe00c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:34.141548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c865f-0b28-47c8-b7ed-3209992fe00c None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:34.142789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c865f-0b28-47c8-b7ed-3209992fe00c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:34.143182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c865f-0b28-47c8-b7ed-3209992fe00c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:34.145381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c865f-0b28-47c8-b7ed-3209992fe00c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:34.159159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:34.169278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 633/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Jul 2 03:24:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:34.175143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52d605-587f-481f-a13f-35e0c9325c95 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:34.279323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52d605-587f-481f-a13f-35e0c9325c95 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:34.314523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb76e9-cd03-4ab0-b927-8791fac27792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:34.315328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb76e9-cd03-4ab0-b927-8791fac27792 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:34.315613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb76e9-cd03-4ab0-b927-8791fac27792 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:34.315881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb76e9-cd03-4ab0-b927-8791fac27792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:34.326817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb76e9-cd03-4ab0-b927-8791fac27792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:34.328294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb76e9-cd03-4ab0-b927-8791fac27792 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-97cb76e9-cd03-4ab0-b927-8791fac27792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:34.345091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:34.346153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:34.347205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:34.347205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:34.347205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:34.347205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:34.347205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 636/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:34.352108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c865f-0b28-47c8-b7ed-3209992fe00c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:34.457476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c865f-0b28-47c8-b7ed-3209992fe00c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:34.493452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a365617-1786-4c96-b9d5-5f38d9a3f567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:34.494105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a365617-1786-4c96-b9d5-5f38d9a3f567 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:34.494385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a365617-1786-4c96-b9d5-5f38d9a3f567 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:34.494661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a365617-1786-4c96-b9d5-5f38d9a3f567 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:34.495406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a365617-1786-4c96-b9d5-5f38d9a3f567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:34.495710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a365617-1786-4c96-b9d5-5f38d9a3f567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:34.497083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a365617-1786-4c96-b9d5-5f38d9a3f567 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:34.508453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 634/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:34.515026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb76e9-cd03-4ab0-b927-8791fac27792 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:34.550287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:34.625719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb76e9-cd03-4ab0-b927-8791fac27792 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:34.663463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df3781-910c-40d2-a208-4a9e273943ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:34.664239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df3781-910c-40d2-a208-4a9e273943ab None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:34.664523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df3781-910c-40d2-a208-4a9e273943ab None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:34.664793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df3781-910c-40d2-a208-4a9e273943ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:34.671957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df3781-910c-40d2-a208-4a9e273943ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:34.673417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df3781-910c-40d2-a208-4a9e273943ab None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-00df3781-910c-40d2-a208-4a9e273943ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:24:34.689327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:34.690324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:34.691237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:24:34.691237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:24:34.691237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:24:34.691237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:24:34.691237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:24:34.691237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:24:34.691237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:34.691237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:34.691237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 637/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:34.696649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a365617-1786-4c96-b9d5-5f38d9a3f567 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:34.759711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:34.815340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a365617-1786-4c96-b9d5-5f38d9a3f567 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:34.853550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3676b-7e45-43fe-8b9f-21d27f788690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:34.854203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3676b-7e45-43fe-8b9f-21d27f788690 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:34.854481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3676b-7e45-43fe-8b9f-21d27f788690 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:34.854751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3676b-7e45-43fe-8b9f-21d27f788690 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:34.855550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3676b-7e45-43fe-8b9f-21d27f788690 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:34.855861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3676b-7e45-43fe-8b9f-21d27f788690 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:34.857299 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3676b-7e45-43fe-8b9f-21d27f788690 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:34.870609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 635/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:34.878192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df3781-910c-40d2-a208-4a9e273943ab None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:34.978257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:34.989595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df3781-910c-40d2-a208-4a9e273943ab None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:35.027134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddc6f1-bbb2-4da7-976b-1d26989aa34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:35.028541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc6f1-bbb2-4da7-976b-1d26989aa34c None admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:35.028907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc6f1-bbb2-4da7-976b-1d26989aa34c None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:35.029248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddc6f1-bbb2-4da7-976b-1d26989aa34c None admin] PATH_INFO: `/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/6ee7acbd4c5545c1885faffb71f6306b/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:35.032734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddc6f1-bbb2-4da7-976b-1d26989aa34c None admin] RBAC: Authorizing `identity:create_grant(project_id=d9882d1720c04bd2aa92db3170e36582, user_id=6ee7acbd4c5545c1885faffb71f6306b, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:35.035024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddc6f1-bbb2-4da7-976b-1d26989aa34c None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:35.052064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 638/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Sat Jul 2 03:24:34 2022] PUT /identity/v3/projects/d9882d1720c04bd2aa92db3170e36582/users/6ee7acbd4c5545c1885faffb71f6306b/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:24:37.093933 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b53f6-25b0-484c-a3f1-3acd7e946827 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:37.094528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b53f6-25b0-484c-a3f1-3acd7e946827 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:37.094528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b53f6-25b0-484c-a3f1-3acd7e946827 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:37.095939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 636/1274] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:37.102778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af6c9-504b-4db7-8234-1635bdb16537 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:37.103589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af6c9-504b-4db7-8234-1635bdb16537 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:37.103752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af6c9-504b-4db7-8234-1635bdb16537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:37.132015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f86af6c9-504b-4db7-8234-1635bdb16537 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:37.133217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f86af6c9-504b-4db7-8234-1635bdb16537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:37.207663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 639/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:24:37.231281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b53f6-25b0-484c-a3f1-3acd7e946827 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:37.337375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b53f6-25b0-484c-a3f1-3acd7e946827 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:37.354974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:37.373794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a199-b85e-405e-9053-415b3d2f707c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:37.374576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a199-b85e-405e-9053-415b3d2f707c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:37.374856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a199-b85e-405e-9053-415b3d2f707c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:37.375130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a199-b85e-405e-9053-415b3d2f707c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:37.386062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5a199-b85e-405e-9053-415b3d2f707c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f5b5a199-b85e-405e-9053-415b3d2f707c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 02 03:24:37.393682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:37.394736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:37.395942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 637/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Sat Jul 2 03:24:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:37.403680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86af6c9-504b-4db7-8234-1635bdb16537 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:37.496516 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:37.524252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86af6c9-504b-4db7-8234-1635bdb16537 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:37.572055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516175c3-f1e1-4e6b-87d0-6e9bb75cb1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:37.572716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516175c3-f1e1-4e6b-87d0-6e9bb75cb1ee None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:37.572996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516175c3-f1e1-4e6b-87d0-6e9bb75cb1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:37.573269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516175c3-f1e1-4e6b-87d0-6e9bb75cb1ee None admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:37.574011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516175c3-f1e1-4e6b-87d0-6e9bb75cb1ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:37.574317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516175c3-f1e1-4e6b-87d0-6e9bb75cb1ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:37.575723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516175c3-f1e1-4e6b-87d0-6e9bb75cb1ee None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:37.584899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 640/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Sat Jul 2 03:24:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:37.590901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a199-b85e-405e-9053-415b3d2f707c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:37.707359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a199-b85e-405e-9053-415b3d2f707c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:37.746001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3992a-5b0e-441c-afad-caf03ddb5e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:37.747121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3992a-5b0e-441c-afad-caf03ddb5e32 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:37.747495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3992a-5b0e-441c-afad-caf03ddb5e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:37.747844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3992a-5b0e-441c-afad-caf03ddb5e32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:37.761629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3992a-5b0e-441c-afad-caf03ddb5e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:37.763081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3992a-5b0e-441c-afad-caf03ddb5e32 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a6c3992a-5b0e-441c-afad-caf03ddb5e32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:37.780734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:37.781753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:37.782669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:37.782669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:37.782669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:37.782669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:37.782669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 638/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:37.789560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516175c3-f1e1-4e6b-87d0-6e9bb75cb1ee None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:37.933639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516175c3-f1e1-4e6b-87d0-6e9bb75cb1ee None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:37.975885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc52253-2c6e-4cd2-a4f2-5d76ac1e6caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:37.976471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc52253-2c6e-4cd2-a4f2-5d76ac1e6caf None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:37.976754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc52253-2c6e-4cd2-a4f2-5d76ac1e6caf None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:37.977027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc52253-2c6e-4cd2-a4f2-5d76ac1e6caf None admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:37.977744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc52253-2c6e-4cd2-a4f2-5d76ac1e6caf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:37.978050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc52253-2c6e-4cd2-a4f2-5d76ac1e6caf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:37.979470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc52253-2c6e-4cd2-a4f2-5d76ac1e6caf None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:37.991530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 641/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:24:37.997602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3992a-5b0e-441c-afad-caf03ddb5e32 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:38.029669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:38.103531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3992a-5b0e-441c-afad-caf03ddb5e32 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:38.141185 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70bcd2-8f54-4c1d-ad84-b31bf2f4aab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:38.141987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70bcd2-8f54-4c1d-ad84-b31bf2f4aab4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:38.142267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70bcd2-8f54-4c1d-ad84-b31bf2f4aab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:38.142538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70bcd2-8f54-4c1d-ad84-b31bf2f4aab4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:38.152000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70bcd2-8f54-4c1d-ad84-b31bf2f4aab4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:38.154240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70bcd2-8f54-4c1d-ad84-b31bf2f4aab4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5e70bcd2-8f54-4c1d-ad84-b31bf2f4aab4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 02 03:24:38.173874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:38.175103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:38.178270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 639/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:38.184233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc52253-2c6e-4cd2-a4f2-5d76ac1e6caf None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:38.190274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:38.294955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc52253-2c6e-4cd2-a4f2-5d76ac1e6caf None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:38.330414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b0d57-958f-477e-8253-15b642e58518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:38.331075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b0d57-958f-477e-8253-15b642e58518 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:38.331385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b0d57-958f-477e-8253-15b642e58518 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:38.331669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b0d57-958f-477e-8253-15b642e58518 None admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:38.332552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b0d57-958f-477e-8253-15b642e58518 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:38.332866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b0d57-958f-477e-8253-15b642e58518 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:38.334320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b0d57-958f-477e-8253-15b642e58518 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:38.351203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 642/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Sat Jul 2 03:24:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:24:38.360423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70bcd2-8f54-4c1d-ad84-b31bf2f4aab4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:38.458925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:38.473798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70bcd2-8f54-4c1d-ad84-b31bf2f4aab4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:38.517207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbe96b-7215-4089-ba45-3c90006c2110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:38.517863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbe96b-7215-4089-ba45-3c90006c2110 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:38.518137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbe96b-7215-4089-ba45-3c90006c2110 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:38.518404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbe96b-7215-4089-ba45-3c90006c2110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:38.593967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbe96b-7215-4089-ba45-3c90006c2110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:38.595346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbe96b-7215-4089-ba45-3c90006c2110 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:38.684241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:38.694769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 640/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Sat Jul 2 03:24:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 02 03:24:38.700781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b0d57-958f-477e-8253-15b642e58518 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:38.767818 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:38.813252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b0d57-958f-477e-8253-15b642e58518 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:38.850674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf7720-b8f1-4aa4-97b5-ae20eabda861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:38.851415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf7720-b8f1-4aa4-97b5-ae20eabda861 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:38.851692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf7720-b8f1-4aa4-97b5-ae20eabda861 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:38.851961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf7720-b8f1-4aa4-97b5-ae20eabda861 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:38.862966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf7720-b8f1-4aa4-97b5-ae20eabda861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:38.864513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf7720-b8f1-4aa4-97b5-ae20eabda861 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-07cf7720-b8f1-4aa4-97b5-ae20eabda861 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 02 03:24:38.884322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:38.885288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:38.886227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 02 03:24:38.886227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 02 03:24:38.886227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 02 03:24:38.886227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:38.886227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 643/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:38.891981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbe96b-7215-4089-ba45-3c90006c2110 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:38.996474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbe96b-7215-4089-ba45-3c90006c2110 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:39.031648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161ef4c-7e6b-40ca-8b32-4f5ee5ee662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:39.032292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161ef4c-7e6b-40ca-8b32-4f5ee5ee662c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:39.032572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161ef4c-7e6b-40ca-8b32-4f5ee5ee662c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:39.032845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161ef4c-7e6b-40ca-8b32-4f5ee5ee662c None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:39.033553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161ef4c-7e6b-40ca-8b32-4f5ee5ee662c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:39.033863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161ef4c-7e6b-40ca-8b32-4f5ee5ee662c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:39.035217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161ef4c-7e6b-40ca-8b32-4f5ee5ee662c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:39.046474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 641/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Sat Jul 2 03:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:39.052885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf7720-b8f1-4aa4-97b5-ae20eabda861 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:39.163177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf7720-b8f1-4aa4-97b5-ae20eabda861 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:39.177414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:39.199341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a713a-63c5-440f-81aa-20465f0bf9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:39.200139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a713a-63c5-440f-81aa-20465f0bf9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:39.200417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a713a-63c5-440f-81aa-20465f0bf9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:39.200692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a713a-63c5-440f-81aa-20465f0bf9d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:39.207652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a713a-63c5-440f-81aa-20465f0bf9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:39.209104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a713a-63c5-440f-81aa-20465f0bf9d4 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application [None req-988a713a-63c5-440f-81aa-20465f0bf9d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 02 03:24:39.222732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:39.224139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:39.225220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 02 03:24:39.225220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 02 03:24:39.225220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 02 03:24:39.225220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 02 03:24:39.225220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 02 03:24:39.225220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 02 03:24:39.225220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 02 03:24:39.225220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: ERROR keystone.server.flask.application Jul 02 03:24:39.225220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 644/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Sat Jul 2 03:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:39.230334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161ef4c-7e6b-40ca-8b32-4f5ee5ee662c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:39.282227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:39.346898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161ef4c-7e6b-40ca-8b32-4f5ee5ee662c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:39.381112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996853a-bb81-4c10-b8c2-cf3e5164cba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:39.381687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996853a-bb81-4c10-b8c2-cf3e5164cba2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:39.381962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996853a-bb81-4c10-b8c2-cf3e5164cba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:39.382233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996853a-bb81-4c10-b8c2-cf3e5164cba2 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:39.383015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996853a-bb81-4c10-b8c2-cf3e5164cba2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:39.383339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996853a-bb81-4c10-b8c2-cf3e5164cba2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:39.384768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996853a-bb81-4c10-b8c2-cf3e5164cba2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:39.396709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 642/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Sat Jul 2 03:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:39.406228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a713a-63c5-440f-81aa-20465f0bf9d4 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:39.514014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a713a-63c5-440f-81aa-20465f0bf9d4 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:39.550479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f010c34-9006-4602-a208-1cdb0da368a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:39.551119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f010c34-9006-4602-a208-1cdb0da368a3 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:39.551463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f010c34-9006-4602-a208-1cdb0da368a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:39.551711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f010c34-9006-4602-a208-1cdb0da368a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:39.552589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f010c34-9006-4602-a208-1cdb0da368a3 None admin] RBAC: Adding query filter params (role.id=d5aa55cbd9374aa2a998f4531b9f0eb0, scope.project.id=d9882d1720c04bd2aa92db3170e36582, user.id=6ee7acbd4c5545c1885faffb71f6306b) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:39.552896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f010c34-9006-4602-a208-1cdb0da368a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:39.554334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f010c34-9006-4602-a208-1cdb0da368a3 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:39.561125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 645/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Sat Jul 2 03:24:39 2022] GET /identity/v3/role_assignments?user.id=6ee7acbd4c5545c1885faffb71f6306b&scope.project.id=d9882d1720c04bd2aa92db3170e36582&role.id=d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:24:41.584390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7cb13-5cb6-4f39-a219-eed5a3531da0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:41.584950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7cb13-5cb6-4f39-a219-eed5a3531da0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:41.584950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7cb13-5cb6-4f39-a219-eed5a3531da0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:41.586314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 643/1288] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:41.591600 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee840840-98ae-4550-b617-f46406f52fed None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:41.592047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee840840-98ae-4550-b617-f46406f52fed None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:41.592140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee840840-98ae-4550-b617-f46406f52fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:41.611564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ee840840-98ae-4550-b617-f46406f52fed None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:41.612723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ee840840-98ae-4550-b617-f46406f52fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:41.644641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:41.687408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 646/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:24:41.709707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7cb13-5cb6-4f39-a219-eed5a3531da0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:41.820691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7cb13-5cb6-4f39-a219-eed5a3531da0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:41.831200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:41.858056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d79ba9-3ecf-4bb2-920b-b6cadb066a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:41.858781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d79ba9-3ecf-4bb2-920b-b6cadb066a73 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:41.859060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d79ba9-3ecf-4bb2-920b-b6cadb066a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:41.859336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d79ba9-3ecf-4bb2-920b-b6cadb066a73 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:41.860000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d79ba9-3ecf-4bb2-920b-b6cadb066a73 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:41.861360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d79ba9-3ecf-4bb2-920b-b6cadb066a73 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-42d79ba9-3ecf-4bb2-920b-b6cadb066a73 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:41.873864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:41.875048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:41.875945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 644/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:24:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:41.881670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee840840-98ae-4550-b617-f46406f52fed None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:41.998672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee840840-98ae-4550-b617-f46406f52fed None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:42.033294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091268b9-e812-4134-b579-a087685ccd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:42.033937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091268b9-e812-4134-b579-a087685ccd30 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:42.034220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091268b9-e812-4134-b579-a087685ccd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:42.034489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091268b9-e812-4134-b579-a087685ccd30 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:42.035185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091268b9-e812-4134-b579-a087685ccd30 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:42.035545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091268b9-e812-4134-b579-a087685ccd30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:42.036908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091268b9-e812-4134-b579-a087685ccd30 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:42.041829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 647/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Sat Jul 2 03:24:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:42.048748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d79ba9-3ecf-4bb2-920b-b6cadb066a73 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:42.163742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d79ba9-3ecf-4bb2-920b-b6cadb066a73 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:42.198666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f8c65-2add-4d01-b650-647901a5b57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:42.199346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8c65-2add-4d01-b650-647901a5b57d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:42.199618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8c65-2add-4d01-b650-647901a5b57d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:42.199892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f8c65-2add-4d01-b650-647901a5b57d None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:42.200584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f8c65-2add-4d01-b650-647901a5b57d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:42.200887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f8c65-2add-4d01-b650-647901a5b57d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:42.202258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f8c65-2add-4d01-b650-647901a5b57d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:42.206834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 645/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Sat Jul 2 03:24:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:44.216194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2854640-24ef-4dec-bd5b-43ba2a2cf078 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:44.217081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2854640-24ef-4dec-bd5b-43ba2a2cf078 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:44.217081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2854640-24ef-4dec-bd5b-43ba2a2cf078 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:44.219275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 648/1293] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:44.226031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749befc-1027-4e89-9015-c0bed2eaac5e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:44.226487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749befc-1027-4e89-9015-c0bed2eaac5e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:44.226604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749befc-1027-4e89-9015-c0bed2eaac5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:44.249505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2749befc-1027-4e89-9015-c0bed2eaac5e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:44.250691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2749befc-1027-4e89-9015-c0bed2eaac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:44.286874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:44.338464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 646/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 02 03:24:44.367626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2854640-24ef-4dec-bd5b-43ba2a2cf078 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:44.408537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:44.486677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2854640-24ef-4dec-bd5b-43ba2a2cf078 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:44.521804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fe321-75ea-45cd-9647-e2937a51b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:44.522453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fe321-75ea-45cd-9647-e2937a51b283 None admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:44.522734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fe321-75ea-45cd-9647-e2937a51b283 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:44.523010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fe321-75ea-45cd-9647-e2937a51b283 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:44.523765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0fe321-75ea-45cd-9647-e2937a51b283 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:44.525151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0fe321-75ea-45cd-9647-e2937a51b283 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:44.539779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 649/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Sat Jul 2 03:24:44 2022] POST /identity/v3/services => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:24:46.544074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c880897-3a91-4adf-88b1-141779959028 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:46.544958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c880897-3a91-4adf-88b1-141779959028 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:46.544958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c880897-3a91-4adf-88b1-141779959028 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:46.547147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 647/1296] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:46.555360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aab6be-be5c-406c-8917-4c5b7eb24b6e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:46.556039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aab6be-be5c-406c-8917-4c5b7eb24b6e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:46.556469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aab6be-be5c-406c-8917-4c5b7eb24b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:46.584066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-87aab6be-be5c-406c-8917-4c5b7eb24b6e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:46.585252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-87aab6be-be5c-406c-8917-4c5b7eb24b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:46.663211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 650/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:24:46.687506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c880897-3a91-4adf-88b1-141779959028 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:46.796834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c880897-3a91-4adf-88b1-141779959028 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:46.832201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbb8bb-d8f9-43ba-bdfc-1c9ef76be7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:46.832992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb8bb-d8f9-43ba-bdfc-1c9ef76be7ed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:46.833274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb8bb-d8f9-43ba-bdfc-1c9ef76be7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:46.833541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb8bb-d8f9-43ba-bdfc-1c9ef76be7ed None admin] PATH_INFO: `/v3/services/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:46.834190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbb8bb-d8f9-43ba-bdfc-1c9ef76be7ed None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:46.835600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbb8bb-d8f9-43ba-bdfc-1c9ef76be7ed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-50cbb8bb-d8f9-43ba-bdfc-1c9ef76be7ed None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:46.847374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:46.850118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:46.851132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 648/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:24:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:46.856684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aab6be-be5c-406c-8917-4c5b7eb24b6e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:46.884169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:46.985677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aab6be-be5c-406c-8917-4c5b7eb24b6e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:47.019775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ced02-9156-4d3b-9271-205755fc6f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:47.020424 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ced02-9156-4d3b-9271-205755fc6f78 None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:47.020703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ced02-9156-4d3b-9271-205755fc6f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:47.020978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ced02-9156-4d3b-9271-205755fc6f78 None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:47.021672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ced02-9156-4d3b-9271-205755fc6f78 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:47.021974 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ced02-9156-4d3b-9271-205755fc6f78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:47.023339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ced02-9156-4d3b-9271-205755fc6f78 None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:47.028638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 651/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Sat Jul 2 03:24:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:47.035353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbb8bb-d8f9-43ba-bdfc-1c9ef76be7ed None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:47.082032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:47.157836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbb8bb-d8f9-43ba-bdfc-1c9ef76be7ed None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:47.197623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0213a8-4154-4673-b285-19288de4e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:47.198284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0213a8-4154-4673-b285-19288de4e814 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:47.198566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0213a8-4154-4673-b285-19288de4e814 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:47.198841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0213a8-4154-4673-b285-19288de4e814 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:47.199664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0213a8-4154-4673-b285-19288de4e814 None admin] RBAC: Adding query filter params (interface=public, service_id=f21674e359cb425da0a9aa8a7feeedbe, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:47.199976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0213a8-4154-4673-b285-19288de4e814 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:47.201356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0213a8-4154-4673-b285-19288de4e814 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:47.207034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 649/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Sat Jul 2 03:24:47 2022] GET /identity/v3/endpoints?service_id=f21674e359cb425da0a9aa8a7feeedbe&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:47.213121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ced02-9156-4d3b-9271-205755fc6f78 None admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:47.318349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:47.340998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ced02-9156-4d3b-9271-205755fc6f78 None admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:47.378090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1da17-7292-4c0e-a23f-7d3b85217e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:47.378742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1da17-7292-4c0e-a23f-7d3b85217e2d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:47.379021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1da17-7292-4c0e-a23f-7d3b85217e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:47.379337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1da17-7292-4c0e-a23f-7d3b85217e2d None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:47.380021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1da17-7292-4c0e-a23f-7d3b85217e2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:47.381392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1da17-7292-4c0e-a23f-7d3b85217e2d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:47.388196 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 652/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Sat Jul 2 03:24:47 2022] GET /identity/v3/services => generated 2191 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:24:49.416835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9daf7cf-b252-4f93-a1f7-6992d2f0a991 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:49.417659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9daf7cf-b252-4f93-a1f7-6992d2f0a991 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:49.417659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9daf7cf-b252-4f93-a1f7-6992d2f0a991 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:49.419827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 650/1302] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:24:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:24:49.429188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf44c-f29b-4f8b-a3b0-e951e4c6013c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:49.429922 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf44c-f29b-4f8b-a3b0-e951e4c6013c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:49.430095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bf44c-f29b-4f8b-a3b0-e951e4c6013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:49.458625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c49bf44c-f29b-4f8b-a3b0-e951e4c6013c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:24:49.459838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c49bf44c-f29b-4f8b-a3b0-e951e4c6013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:24:49.536936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 653/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:24:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 02 03:24:49.565803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9daf7cf-b252-4f93-a1f7-6992d2f0a991 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:49.667533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:49.675213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9daf7cf-b252-4f93-a1f7-6992d2f0a991 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:49.724696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685b63f-cc9b-4fbb-9176-28c4ddf7f07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:49.725496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685b63f-cc9b-4fbb-9176-28c4ddf7f07b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:49.725776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685b63f-cc9b-4fbb-9176-28c4ddf7f07b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:49.726050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685b63f-cc9b-4fbb-9176-28c4ddf7f07b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:49.726701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685b63f-cc9b-4fbb-9176-28c4ddf7f07b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:49.728111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685b63f-cc9b-4fbb-9176-28c4ddf7f07b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5685b63f-cc9b-4fbb-9176-28c4ddf7f07b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 02 03:24:49.740506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 02 03:24:49.741625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jul 02 03:24:49.742514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 651/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Sat Jul 2 03:24:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 02 03:24:49.748092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bf44c-f29b-4f8b-a3b0-e951e4c6013c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:49.832273 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:24:49.856597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bf44c-f29b-4f8b-a3b0-e951e4c6013c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:49.891747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d22a7-0ea8-4949-bfc1-7153af02c77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:49.892406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d22a7-0ea8-4949-bfc1-7153af02c77d None admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:49.892687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d22a7-0ea8-4949-bfc1-7153af02c77d None admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:49.892964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d22a7-0ea8-4949-bfc1-7153af02c77d None admin] PATH_INFO: `/v3/services` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:49.893664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d22a7-0ea8-4949-bfc1-7153af02c77d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:24:49.893969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d22a7-0ea8-4949-bfc1-7153af02c77d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:49.895429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d22a7-0ea8-4949-bfc1-7153af02c77d None admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:49.900830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 654/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Sat Jul 2 03:24:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:24:49.906845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685b63f-cc9b-4fbb-9176-28c4ddf7f07b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:24:50.011434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685b63f-cc9b-4fbb-9176-28c4ddf7f07b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:24:50.050269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdfd04-f085-4fef-beeb-b44ae6f17b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:24:50.050939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdfd04-f085-4fef-beeb-b44ae6f17b8e None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:24:50.051220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdfd04-f085-4fef-beeb-b44ae6f17b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:24:50.051521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdfd04-f085-4fef-beeb-b44ae6f17b8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:24:50.052253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdfd04-f085-4fef-beeb-b44ae6f17b8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:24:50.053634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdfd04-f085-4fef-beeb-b44ae6f17b8e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:24:50.075760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 652/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Sat Jul 2 03:24:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:24:59.898143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=30 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:25:00.070883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:25:53.188490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afcad4-775b-4e8e-9497-adf335e20661 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:53.189499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afcad4-775b-4e8e-9497-adf335e20661 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:53.190000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afcad4-775b-4e8e-9497-adf335e20661 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:53.197499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d51f7e-af42-4dc3-acd6-7b138ba1a713 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:53.197919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d51f7e-af42-4dc3-acd6-7b138ba1a713 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:53.198241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d51f7e-af42-4dc3-acd6-7b138ba1a713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:53.200722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 655/1307] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:25:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:25:53.291335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-98d51f7e-af42-4dc3-acd6-7b138ba1a713 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:25:53.293135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-98d51f7e-af42-4dc3-acd6-7b138ba1a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:25:53.493302 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 653/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:25:53.535953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d951774-d6aa-435f-91ee-c8ac5f36c4d6 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:53.536772 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d951774-d6aa-435f-91ee-c8ac5f36c4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:53.537449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d951774-d6aa-435f-91ee-c8ac5f36c4d6 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:53.538912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 656/1309] 10.222.0.42 () {54 vars in 964 bytes} [Sat Jul 2 03:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:25:53.545450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de91e39-1a17-4437-a10c-e5432ff47873 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:53.545665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de91e39-1a17-4437-a10c-e5432ff47873 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:53.545949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de91e39-1a17-4437-a10c-e5432ff47873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:53.637101 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1de91e39-1a17-4437-a10c-e5432ff47873 None None] MFA Rules not processed for user `d67be26446324c91b956e9d930e04b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:25:53.638320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1de91e39-1a17-4437-a10c-e5432ff47873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:25:53.660140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:25:53.720730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 654/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Sat Jul 2 03:25:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:25:53.730464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d951774-d6aa-435f-91ee-c8ac5f36c4d6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:25:53.732175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3d951774-d6aa-435f-91ee-c8ac5f36c4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:25:53.849201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d951774-d6aa-435f-91ee-c8ac5f36c4d6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:25:53.926146 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7d480-9179-49e1-9b65-4683c8058fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:25:53.926799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7d480-9179-49e1-9b65-4683c8058fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:53.927129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7d480-9179-49e1-9b65-4683c8058fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:53.927532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7d480-9179-49e1-9b65-4683c8058fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:53.929377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9bc7d480-9179-49e1-9b65-4683c8058fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:25:53.986124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:25:54.047904 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7d480-9179-49e1-9b65-4683c8058fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:25:54.049379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7d480-9179-49e1-9b65-4683c8058fb6 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:25:54.070893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 657/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:25:56.418298 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796a028-d45f-4d48-8c00-eec208fe2869 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:56.418872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796a028-d45f-4d48-8c00-eec208fe2869 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:56.418872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796a028-d45f-4d48-8c00-eec208fe2869 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:56.420467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 655/1312] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:25:56.426006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af902a1a-4ec5-4513-a306-b2582ed2d34c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:56.426457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af902a1a-4ec5-4513-a306-b2582ed2d34c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:56.426565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af902a1a-4ec5-4513-a306-b2582ed2d34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:56.516113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-af902a1a-4ec5-4513-a306-b2582ed2d34c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:25:56.517432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-af902a1a-4ec5-4513-a306-b2582ed2d34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:25:56.630191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 658/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:25:56.683881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f10532-075c-4bb0-a359-ef27942c70d8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:56.684088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f10532-075c-4bb0-a359-ef27942c70d8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:56.684443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f10532-075c-4bb0-a359-ef27942c70d8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:56.686067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 656/1314] 10.222.0.42 () {54 vars in 964 bytes} [Sat Jul 2 03:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:25:56.693113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635941e9-2627-4063-9b74-4253fb8da678 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:56.693406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635941e9-2627-4063-9b74-4253fb8da678 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:56.693685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635941e9-2627-4063-9b74-4253fb8da678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:56.704318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:25:56.776727 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-635941e9-2627-4063-9b74-4253fb8da678 None None] MFA Rules not processed for user `d67be26446324c91b956e9d930e04b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:25:56.777955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-635941e9-2627-4063-9b74-4253fb8da678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:25:56.866043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 659/1315] 10.222.0.42 () {58 vars in 1061 bytes} [Sat Jul 2 03:25:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:25:56.874701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f10532-075c-4bb0-a359-ef27942c70d8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:25:56.876276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-98f10532-075c-4bb0-a359-ef27942c70d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:25:56.982663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f10532-075c-4bb0-a359-ef27942c70d8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:25:57.058949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:25:57.061875 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab475485-aaf4-409d-a481-45143eed6c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:25:57.062546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475485-aaf4-409d-a481-45143eed6c8e service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:57.063204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475485-aaf4-409d-a481-45143eed6c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:57.063732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab475485-aaf4-409d-a481-45143eed6c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:57.065595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ab475485-aaf4-409d-a481-45143eed6c8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:25:57.183201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab475485-aaf4-409d-a481-45143eed6c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:25:57.184679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab475485-aaf4-409d-a481-45143eed6c8e service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:25:57.203160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 657/1316] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:25:59.719620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d36a6-138c-491d-9e13-846a09f4b0d4 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:59.720483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d36a6-138c-491d-9e13-846a09f4b0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:59.720483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d36a6-138c-491d-9e13-846a09f4b0d4 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:59.722641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 660/1317] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:25:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:25:59.729801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fbb17-22b1-4242-97b9-6abf30e7e0c2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:25:59.730230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fbb17-22b1-4242-97b9-6abf30e7e0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:25:59.730398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fbb17-22b1-4242-97b9-6abf30e7e0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:25:59.805286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-506fbb17-22b1-4242-97b9-6abf30e7e0c2 None None] MFA Rules not processed for user `80cfaef5476c4d42b916f2cf9c155962`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:25:59.806837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-506fbb17-22b1-4242-97b9-6abf30e7e0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:25:59.870986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:25:59.919394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 658/1318] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:25:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:25:59.958936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d36a6-138c-491d-9e13-846a09f4b0d4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:25:59.960572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ea3d36a6-138c-491d-9e13-846a09f4b0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:00.070228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d36a6-138c-491d-9e13-846a09f4b0d4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:00.073079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:00.147840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022228d-54dc-4daf-843c-7133338b9804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:00.148486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022228d-54dc-4daf-843c-7133338b9804 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:00.148771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022228d-54dc-4daf-843c-7133338b9804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:00.149051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022228d-54dc-4daf-843c-7133338b9804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:00.150685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8022228d-54dc-4daf-843c-7133338b9804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:00.260201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022228d-54dc-4daf-843c-7133338b9804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:00.261652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022228d-54dc-4daf-843c-7133338b9804 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:00.281497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 661/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:25:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:03.023942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387e97d-f937-444f-9e27-d35d98fc4766 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:03.024461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387e97d-f937-444f-9e27-d35d98fc4766 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:03.024461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387e97d-f937-444f-9e27-d35d98fc4766 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:03.025899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 659/1320] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:03.032250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4841fb-0ee4-4e7d-b49e-3824c286468e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:03.033102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4841fb-0ee4-4e7d-b49e-3824c286468e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:03.033102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4841fb-0ee4-4e7d-b49e-3824c286468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:03.081714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:03.111117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-da4841fb-0ee4-4e7d-b49e-3824c286468e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:03.112405 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-da4841fb-0ee4-4e7d-b49e-3824c286468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:03.213840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 662/1321] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:03.252008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387e97d-f937-444f-9e27-d35d98fc4766 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:03.253533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0387e97d-f937-444f-9e27-d35d98fc4766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:03.377695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0387e97d-f937-444f-9e27-d35d98fc4766 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:03.456004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d19340-fc6a-48c5-b335-f061211f9f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:03.457018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d19340-fc6a-48c5-b335-f061211f9f9d service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:03.457378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d19340-fc6a-48c5-b335-f061211f9f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:03.457822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d19340-fc6a-48c5-b335-f061211f9f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:03.460699 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-46d19340-fc6a-48c5-b335-f061211f9f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:03.473150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:03.583640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d19340-fc6a-48c5-b335-f061211f9f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:03.585089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d19340-fc6a-48c5-b335-f061211f9f9d service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:03.603268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 660/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:06.368138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef0d20-7edf-4061-a0ad-5ed58f3d8218 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:06.368663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef0d20-7edf-4061-a0ad-5ed58f3d8218 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:06.368663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef0d20-7edf-4061-a0ad-5ed58f3d8218 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:06.370003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 663/1323] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:06.375639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af10ee2-6c12-4d09-9bc4-2f2574bb57c4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:06.376082 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af10ee2-6c12-4d09-9bc4-2f2574bb57c4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:06.376314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af10ee2-6c12-4d09-9bc4-2f2574bb57c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:06.446696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9af10ee2-6c12-4d09-9bc4-2f2574bb57c4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:06.447934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9af10ee2-6c12-4d09-9bc4-2f2574bb57c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:06.512355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:06.555542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 661/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:06.594465 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef0d20-7edf-4061-a0ad-5ed58f3d8218 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:06.596645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-79ef0d20-7edf-4061-a0ad-5ed58f3d8218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:06.718271 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef0d20-7edf-4061-a0ad-5ed58f3d8218 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:06.731994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:06.799758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62829be-bb2d-4075-9440-012f45c26658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:06.800389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62829be-bb2d-4075-9440-012f45c26658 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:06.800677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62829be-bb2d-4075-9440-012f45c26658 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:06.800951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62829be-bb2d-4075-9440-012f45c26658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:06.803836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d62829be-bb2d-4075-9440-012f45c26658 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:06.919583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62829be-bb2d-4075-9440-012f45c26658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:06.920921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62829be-bb2d-4075-9440-012f45c26658 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:06.939839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 664/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:09.013660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c124fd8-8650-42a7-bfa1-4928d4c7480d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:09.014212 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c124fd8-8650-42a7-bfa1-4928d4c7480d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:09.014212 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c124fd8-8650-42a7-bfa1-4928d4c7480d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:09.015644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 662/1326] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:09.021121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733008a5-2483-4da0-b811-003ee2978429 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:09.021554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733008a5-2483-4da0-b811-003ee2978429 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:09.021671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733008a5-2483-4da0-b811-003ee2978429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:09.071742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:09.091241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-733008a5-2483-4da0-b811-003ee2978429 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:09.092492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-733008a5-2483-4da0-b811-003ee2978429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:09.196826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 665/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:09.232621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c124fd8-8650-42a7-bfa1-4928d4c7480d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:09.234154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9c124fd8-8650-42a7-bfa1-4928d4c7480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:09.339121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c124fd8-8650-42a7-bfa1-4928d4c7480d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:09.418362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedfa5fd-48c1-4f93-bb3a-d6845d3394c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:09.419013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfa5fd-48c1-4f93-bb3a-d6845d3394c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:09.419316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfa5fd-48c1-4f93-bb3a-d6845d3394c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:09.419588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedfa5fd-48c1-4f93-bb3a-d6845d3394c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:09.421235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aedfa5fd-48c1-4f93-bb3a-d6845d3394c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:09.438901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:09.534273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedfa5fd-48c1-4f93-bb3a-d6845d3394c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:09.535993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedfa5fd-48c1-4f93-bb3a-d6845d3394c9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:09.556974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 663/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:11.668726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b591104-f040-4f73-b2f1-acc5423002c3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:11.669284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b591104-f040-4f73-b2f1-acc5423002c3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:11.669284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b591104-f040-4f73-b2f1-acc5423002c3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:11.670694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 666/1329] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:11.676267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b5dfb-48c6-4e11-a15a-bc7b810a0bb4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:11.676716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b5dfb-48c6-4e11-a15a-bc7b810a0bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:11.676950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b5dfb-48c6-4e11-a15a-bc7b810a0bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:11.757887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-745b5dfb-48c6-4e11-a15a-bc7b810a0bb4 None None] MFA Rules not processed for user `80cfaef5476c4d42b916f2cf9c155962`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:11.759169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-745b5dfb-48c6-4e11-a15a-bc7b810a0bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:11.833237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:11.867475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 664/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:11.904663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b591104-f040-4f73-b2f1-acc5423002c3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:11.906212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0b591104-f040-4f73-b2f1-acc5423002c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:12.015184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b591104-f040-4f73-b2f1-acc5423002c3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:12.037408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:12.100991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d613-8614-4782-844f-6fc5a0cce179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:12.101552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d613-8614-4782-844f-6fc5a0cce179 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:12.101835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d613-8614-4782-844f-6fc5a0cce179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:12.102108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d613-8614-4782-844f-6fc5a0cce179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:12.103777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a9c0d613-8614-4782-844f-6fc5a0cce179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:12.227286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0d613-8614-4782-844f-6fc5a0cce179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:12.228740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0d613-8614-4782-844f-6fc5a0cce179 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:12.249292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 667/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:14.545266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27614-a0b0-4f44-bec7-f997a3cdf2e8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:14.545823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27614-a0b0-4f44-bec7-f997a3cdf2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:14.545823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27614-a0b0-4f44-bec7-f997a3cdf2e8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:14.548066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 665/1332] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:14.554509 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676c18a-9e22-4b76-9cfd-b5ae325235d9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:14.555640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676c18a-9e22-4b76-9cfd-b5ae325235d9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:14.555800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676c18a-9e22-4b76-9cfd-b5ae325235d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:14.630898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:14.633999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f676c18a-9e22-4b76-9cfd-b5ae325235d9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:14.635779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f676c18a-9e22-4b76-9cfd-b5ae325235d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:14.734002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 668/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:14.769992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e27614-a0b0-4f44-bec7-f997a3cdf2e8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:14.771571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b6e27614-a0b0-4f44-bec7-f997a3cdf2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:14.889221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e27614-a0b0-4f44-bec7-f997a3cdf2e8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:14.967691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136da467-01ee-46d9-b904-172d76c33f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:14.968353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136da467-01ee-46d9-b904-172d76c33f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:14.968637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136da467-01ee-46d9-b904-172d76c33f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:14.968910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136da467-01ee-46d9-b904-172d76c33f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:14.970617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-136da467-01ee-46d9-b904-172d76c33f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:15.000337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:15.088496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136da467-01ee-46d9-b904-172d76c33f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:15.089953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136da467-01ee-46d9-b904-172d76c33f88 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:15.113583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 666/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:17.197077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44448b51-5b53-4c49-b627-4b816beb1095 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:17.197666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44448b51-5b53-4c49-b627-4b816beb1095 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:17.197666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44448b51-5b53-4c49-b627-4b816beb1095 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:17.199011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 669/1335] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:26:17.204679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9328bb6-961d-4529-8480-25b0f2fed8a0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:17.205122 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9328bb6-961d-4529-8480-25b0f2fed8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:17.205222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9328bb6-961d-4529-8480-25b0f2fed8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:17.279451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b9328bb6-961d-4529-8480-25b0f2fed8a0 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:17.280698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b9328bb6-961d-4529-8480-25b0f2fed8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:17.365112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:17.392906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 667/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:17.437371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44448b51-5b53-4c49-b627-4b816beb1095 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:17.439872 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-44448b51-5b53-4c49-b627-4b816beb1095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:17.563224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44448b51-5b53-4c49-b627-4b816beb1095 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:17.592462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:17.639859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262044b-0df2-4fe5-ad47-cb7a0b0032ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:17.640512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262044b-0df2-4fe5-ad47-cb7a0b0032ed service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:17.640793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262044b-0df2-4fe5-ad47-cb7a0b0032ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:17.641068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262044b-0df2-4fe5-ad47-cb7a0b0032ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:17.642700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d262044b-0df2-4fe5-ad47-cb7a0b0032ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:17.756344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262044b-0df2-4fe5-ad47-cb7a0b0032ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:17.757793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262044b-0df2-4fe5-ad47-cb7a0b0032ed service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:17.777859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 670/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:20.187948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf6512-dce4-4edc-b618-31f7d9a8f2e8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:20.188552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf6512-dce4-4edc-b618-31f7d9a8f2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:20.188552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf6512-dce4-4edc-b618-31f7d9a8f2e8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:20.189894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 668/1338] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:20.195524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f0287-757d-4a4f-9878-33bd63fd3ea6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:20.195969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f0287-757d-4a4f-9878-33bd63fd3ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:20.196074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f0287-757d-4a4f-9878-33bd63fd3ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:20.266058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9a2f0287-757d-4a4f-9878-33bd63fd3ea6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:20.267364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9a2f0287-757d-4a4f-9878-33bd63fd3ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:20.275917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:20.367461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 671/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:20.407556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf6512-dce4-4edc-b618-31f7d9a8f2e8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:20.409961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aabf6512-dce4-4edc-b618-31f7d9a8f2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:20.523967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf6512-dce4-4edc-b618-31f7d9a8f2e8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:20.603663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90c2d7-a283-41c1-bdbb-97a9263cb922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:20.604575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c2d7-a283-41c1-bdbb-97a9263cb922 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:20.605028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c2d7-a283-41c1-bdbb-97a9263cb922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:20.605449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c2d7-a283-41c1-bdbb-97a9263cb922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:20.608241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8f90c2d7-a283-41c1-bdbb-97a9263cb922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:20.656141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:20.731991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90c2d7-a283-41c1-bdbb-97a9263cb922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:20.733436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90c2d7-a283-41c1-bdbb-97a9263cb922 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:20.753432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 669/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:24.938695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598843bc-b5a6-400f-b5ba-7f825f2c56af None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:24.939236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598843bc-b5a6-400f-b5ba-7f825f2c56af None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:24.939236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598843bc-b5a6-400f-b5ba-7f825f2c56af None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:24.940701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 672/1341] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:24.946472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ae756-46bf-4c65-b463-18072983b66d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:24.946940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ae756-46bf-4c65-b463-18072983b66d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:24.947035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ae756-46bf-4c65-b463-18072983b66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:25.025947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2c0ae756-46bf-4c65-b463-18072983b66d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:25.027144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2c0ae756-46bf-4c65-b463-18072983b66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:25.133829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:25.151462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 670/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:25.187713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598843bc-b5a6-400f-b5ba-7f825f2c56af None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:25.189927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-598843bc-b5a6-400f-b5ba-7f825f2c56af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:25.289048 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598843bc-b5a6-400f-b5ba-7f825f2c56af None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:25.292717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a1ef8-9287-40ea-bf9d-adb53689c1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:25.293381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a1ef8-9287-40ea-bf9d-adb53689c1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:25.293663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a1ef8-9287-40ea-bf9d-adb53689c1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:25.293939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a1ef8-9287-40ea-bf9d-adb53689c1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:25.303202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a1ef8-9287-40ea-bf9d-adb53689c1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:25.305095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a1ef8-9287-40ea-bf9d-adb53689c1b8 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:25.314943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 673/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:27.996452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354af59-b1f8-44b5-87b1-b7ee1df88188 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:27.997063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354af59-b1f8-44b5-87b1-b7ee1df88188 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:27.997063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354af59-b1f8-44b5-87b1-b7ee1df88188 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:27.998743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 671/1344] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:28.004270 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdb2be-4484-460b-8a84-0351a3ddf3bf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:28.004703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdb2be-4484-460b-8a84-0351a3ddf3bf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:28.004814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdb2be-4484-460b-8a84-0351a3ddf3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:28.032771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7ffdb2be-4484-460b-8a84-0351a3ddf3bf None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:28.033969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7ffdb2be-4484-460b-8a84-0351a3ddf3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:28.067254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:28.123629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 674/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:28.161812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354af59-b1f8-44b5-87b1-b7ee1df88188 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:28.163780 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7354af59-b1f8-44b5-87b1-b7ee1df88188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:28.177535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354af59-b1f8-44b5-87b1-b7ee1df88188 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:28.181701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdeebc8-a73c-4f7c-8455-f8edbe6d3a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:28.182243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeebc8-a73c-4f7c-8455-f8edbe6d3a4b service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:28.182528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeebc8-a73c-4f7c-8455-f8edbe6d3a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:28.182806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeebc8-a73c-4f7c-8455-f8edbe6d3a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:28.193099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdeebc8-a73c-4f7c-8455-f8edbe6d3a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:28.194475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdeebc8-a73c-4f7c-8455-f8edbe6d3a4b service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:28.205588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 672/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:32.421120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd827a-6d78-4bc8-8e6c-22aa294b6487 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:32.421630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd827a-6d78-4bc8-8e6c-22aa294b6487 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:32.421711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd827a-6d78-4bc8-8e6c-22aa294b6487 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:32.426805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 675/1347] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:32.428945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171f7e8-674d-4fed-8612-6144a0666645 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:32.429786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171f7e8-674d-4fed-8612-6144a0666645 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:32.429981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171f7e8-674d-4fed-8612-6144a0666645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:32.449777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1171f7e8-674d-4fed-8612-6144a0666645 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:32.450986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1171f7e8-674d-4fed-8612-6144a0666645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:32.459697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 673/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:32.497692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd827a-6d78-4bc8-8e6c-22aa294b6487 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:32.509900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd827a-6d78-4bc8-8e6c-22aa294b6487 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:32.514171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213f4b9-e1fc-49da-af36-298d6c056ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:32.514816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213f4b9-e1fc-49da-af36-298d6c056ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:32.515069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213f4b9-e1fc-49da-af36-298d6c056ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:32.515372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213f4b9-e1fc-49da-af36-298d6c056ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:32.525173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213f4b9-e1fc-49da-af36-298d6c056ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:32.526657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213f4b9-e1fc-49da-af36-298d6c056ac8 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:32.536693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 676/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:37.550852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21d4da-b7dc-40c0-b7bb-ace479acbd6f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:37.551483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21d4da-b7dc-40c0-b7bb-ace479acbd6f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:37.551483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21d4da-b7dc-40c0-b7bb-ace479acbd6f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:37.552906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 674/1350] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:37.559109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ee4d7-4c4e-4a01-9b23-6ed6572685fd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:37.559564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ee4d7-4c4e-4a01-9b23-6ed6572685fd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:37.559708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ee4d7-4c4e-4a01-9b23-6ed6572685fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:37.630627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e73ee4d7-4c4e-4a01-9b23-6ed6572685fd None None] MFA Rules not processed for user `6ee7acbd4c5545c1885faffb71f6306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:37.631981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e73ee4d7-4c4e-4a01-9b23-6ed6572685fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:37.708241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 677/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:26:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:42.029016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82684-62e2-44ac-8097-e54fba047993 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:42.029975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82684-62e2-44ac-8097-e54fba047993 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:42.030258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd82684-62e2-44ac-8097-e54fba047993 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:42.031935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 675/1352] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:26:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:42.041804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba575f8-7be2-4f66-9b6b-3a17a3b71131 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:42.042361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba575f8-7be2-4f66-9b6b-3a17a3b71131 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:42.042361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba575f8-7be2-4f66-9b6b-3a17a3b71131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:42.077873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fba575f8-7be2-4f66-9b6b-3a17a3b71131 None None] MFA Rules not processed for user `6ee7acbd4c5545c1885faffb71f6306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:42.080333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fba575f8-7be2-4f66-9b6b-3a17a3b71131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:42.092109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 678/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:26:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:42.443076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=13 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:43.676860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2803193-8a2d-402e-9093-a9d45eec8896 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:43.677465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2803193-8a2d-402e-9093-a9d45eec8896 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:43.677465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2803193-8a2d-402e-9093-a9d45eec8896 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:43.679174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 676/1354] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:43.684633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc2049-f6ab-45a9-a1e6-374ae504613e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:43.685085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc2049-f6ab-45a9-a1e6-374ae504613e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:43.685189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc2049-f6ab-45a9-a1e6-374ae504613e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:43.704632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f3bc2049-f6ab-45a9-a1e6-374ae504613e None None] MFA Rules not processed for user `6ee7acbd4c5545c1885faffb71f6306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:43.705824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f3bc2049-f6ab-45a9-a1e6-374ae504613e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:43.715221 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 679/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:26:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:26:47.472066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb5cef-6816-4a2f-a709-5768f83a1359 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:47.473770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb5cef-6816-4a2f-a709-5768f83a1359 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:47.474077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb5cef-6816-4a2f-a709-5768f83a1359 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:47.479264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 677/1356] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:47.481458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633e0f1-d4b6-45c3-bb93-4be03dece09b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:47.482294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633e0f1-d4b6-45c3-bb93-4be03dece09b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:47.482774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633e0f1-d4b6-45c3-bb93-4be03dece09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:47.506604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5633e0f1-d4b6-45c3-bb93-4be03dece09b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:47.508329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5633e0f1-d4b6-45c3-bb93-4be03dece09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:47.518238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 680/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:47.578370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda465a-b741-4c3f-907d-9f96225669ad None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:47.578939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda465a-b741-4c3f-907d-9f96225669ad None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:47.579472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda465a-b741-4c3f-907d-9f96225669ad None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:47.581152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 678/1358] 10.222.0.42 () {54 vars in 961 bytes} [Sat Jul 2 03:26:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:26:47.591594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa23c7-eef2-4058-83ab-5b0bc6058ad6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:47.592777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa23c7-eef2-4058-83ab-5b0bc6058ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:47.593604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa23c7-eef2-4058-83ab-5b0bc6058ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:47.647156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:47.656932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b9aa23c7-eef2-4058-83ab-5b0bc6058ad6 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:47.658140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b9aa23c7-eef2-4058-83ab-5b0bc6058ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:47.755944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 681/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Sat Jul 2 03:26:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:47.763807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda465a-b741-4c3f-907d-9f96225669ad None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:47.778274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda465a-b741-4c3f-907d-9f96225669ad None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:47.783884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381f6e9-357f-4c39-be4a-a8adba497fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:47.784582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381f6e9-357f-4c39-be4a-a8adba497fca service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:47.784865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381f6e9-357f-4c39-be4a-a8adba497fca service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:47.785137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381f6e9-357f-4c39-be4a-a8adba497fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:47.794044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381f6e9-357f-4c39-be4a-a8adba497fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:47.795676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381f6e9-357f-4c39-be4a-a8adba497fca service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:47.806189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 679/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:50.036551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea689e-97e0-400a-8b0d-282802f59239 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:50.037135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea689e-97e0-400a-8b0d-282802f59239 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:50.037135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea689e-97e0-400a-8b0d-282802f59239 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:50.038687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 682/1361] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:50.044026 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfea1eb-7e62-4dd0-8363-07598ad8bc64 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:50.044474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfea1eb-7e62-4dd0-8363-07598ad8bc64 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:50.044568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfea1eb-7e62-4dd0-8363-07598ad8bc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:50.064285 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fbfea1eb-7e62-4dd0-8363-07598ad8bc64 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:50.066137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fbfea1eb-7e62-4dd0-8363-07598ad8bc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:50.075040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 680/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:50.123849 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea689e-97e0-400a-8b0d-282802f59239 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:50.142747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea689e-97e0-400a-8b0d-282802f59239 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:50.149221 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6f172-737c-40e8-a43d-25ca2ed31e37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:50.149962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6f172-737c-40e8-a43d-25ca2ed31e37 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:50.150322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6f172-737c-40e8-a43d-25ca2ed31e37 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:50.150673 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6f172-737c-40e8-a43d-25ca2ed31e37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:50.160437 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6f172-737c-40e8-a43d-25ca2ed31e37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:50.163124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6f172-737c-40e8-a43d-25ca2ed31e37 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:50.174114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 683/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:52.595965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d309d-a894-4801-8473-12477f44dd16 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:52.596550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d309d-a894-4801-8473-12477f44dd16 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:52.597124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d309d-a894-4801-8473-12477f44dd16 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:52.598570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 681/1364] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:52.604037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b364fd-ff42-449c-b434-e81b2d391e49 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:52.604477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b364fd-ff42-449c-b434-e81b2d391e49 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:52.604568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b364fd-ff42-449c-b434-e81b2d391e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:52.624280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b2b364fd-ff42-449c-b434-e81b2d391e49 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:52.626160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b2b364fd-ff42-449c-b434-e81b2d391e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:52.640618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 684/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:52.690321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef9e3-157c-4905-a6d2-cd686367c476 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:52.690709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef9e3-157c-4905-a6d2-cd686367c476 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:52.690961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fef9e3-157c-4905-a6d2-cd686367c476 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:52.692793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 682/1366] 10.222.0.42 () {54 vars in 961 bytes} [Sat Jul 2 03:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:52.698409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0bf05-73e6-480e-b274-14cc1bd81fe7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:52.698701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0bf05-73e6-480e-b274-14cc1bd81fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:52.698980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0bf05-73e6-480e-b274-14cc1bd81fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:52.720455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-cea0bf05-73e6-480e-b274-14cc1bd81fe7 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:52.721769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cea0bf05-73e6-480e-b274-14cc1bd81fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:52.731206 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 685/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Sat Jul 2 03:26:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:26:52.739694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fef9e3-157c-4905-a6d2-cd686367c476 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:52.754326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fef9e3-157c-4905-a6d2-cd686367c476 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:52.759700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4396e980-7d17-4c66-ae4b-78dde64dedb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:52.760336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396e980-7d17-4c66-ae4b-78dde64dedb0 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:52.760620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396e980-7d17-4c66-ae4b-78dde64dedb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:52.760889 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4396e980-7d17-4c66-ae4b-78dde64dedb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:52.769698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396e980-7d17-4c66-ae4b-78dde64dedb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:52.771329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4396e980-7d17-4c66-ae4b-78dde64dedb0 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:52.781142 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 683/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:54.889892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a359c70-f84f-43c1-b5ee-aadb66312d00 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:54.890447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a359c70-f84f-43c1-b5ee-aadb66312d00 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:54.890447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a359c70-f84f-43c1-b5ee-aadb66312d00 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:54.892015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 686/1369] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:54.897697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3181fb-0b55-40d1-9eef-72b64f685f5d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:54.898190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3181fb-0b55-40d1-9eef-72b64f685f5d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:54.898276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3181fb-0b55-40d1-9eef-72b64f685f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:54.916413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ff3181fb-0b55-40d1-9eef-72b64f685f5d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:54.917605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ff3181fb-0b55-40d1-9eef-72b64f685f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:54.926004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 684/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:54.970037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a359c70-f84f-43c1-b5ee-aadb66312d00 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:54.985030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a359c70-f84f-43c1-b5ee-aadb66312d00 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:54.990676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d5311-bf8a-489d-b0c8-c90dd9551c90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:54.991719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5311-bf8a-489d-b0c8-c90dd9551c90 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:54.991825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5311-bf8a-489d-b0c8-c90dd9551c90 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:54.992105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d5311-bf8a-489d-b0c8-c90dd9551c90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:55.000921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d5311-bf8a-489d-b0c8-c90dd9551c90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:55.002591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d5311-bf8a-489d-b0c8-c90dd9551c90 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:55.012615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 687/1371] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:55.752145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d687a0-8649-4d26-9f4f-ce8b4c76bd0d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:55.752593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d687a0-8649-4d26-9f4f-ce8b4c76bd0d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:55.753400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d687a0-8649-4d26-9f4f-ce8b4c76bd0d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:55.754817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 685/1372] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:55.760518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478da90e-f056-45c0-a569-86e8ba61fa1f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:55.760877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478da90e-f056-45c0-a569-86e8ba61fa1f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:55.761218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478da90e-f056-45c0-a569-86e8ba61fa1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:55.781591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-478da90e-f056-45c0-a569-86e8ba61fa1f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:55.782914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-478da90e-f056-45c0-a569-86e8ba61fa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:55.792522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 688/1373] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:56.113593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fb617-eccb-4e23-a21c-5f9a4b094acd None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:56.114043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fb617-eccb-4e23-a21c-5f9a4b094acd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:56.114329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fb617-eccb-4e23-a21c-5f9a4b094acd None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:56.116213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 686/1374] 10.222.0.42 () {54 vars in 963 bytes} [Sat Jul 2 03:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:56.122375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14014f44-f406-4a35-834a-96c3f7dc4bf1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:56.122824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14014f44-f406-4a35-834a-96c3f7dc4bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:56.122926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14014f44-f406-4a35-834a-96c3f7dc4bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:56.184069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-14014f44-f406-4a35-834a-96c3f7dc4bf1 None None] MFA Rules not processed for user `e02b8a64165541889ca929b8038412bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:56.185773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-14014f44-f406-4a35-834a-96c3f7dc4bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:56.256732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 689/1375] 10.222.0.42 () {58 vars in 1060 bytes} [Sat Jul 2 03:26:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:26:56.264326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fb617-eccb-4e23-a21c-5f9a4b094acd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:56.276330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fb617-eccb-4e23-a21c-5f9a4b094acd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:56.280222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b2fdd-f7f0-45bb-8e57-656f4b383fa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:56.281327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b2fdd-f7f0-45bb-8e57-656f4b383fa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:56.281811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b2fdd-f7f0-45bb-8e57-656f4b383fa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:56.282323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b2fdd-f7f0-45bb-8e57-656f4b383fa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:56.293236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b2fdd-f7f0-45bb-8e57-656f4b383fa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:56.294944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b2fdd-f7f0-45bb-8e57-656f4b383fa1 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:56.320329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 687/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:57.109414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2fb43-bc6b-42cb-8838-310650cd3a64 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:57.109414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2fb43-bc6b-42cb-8838-310650cd3a64 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:57.109414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2fb43-bc6b-42cb-8838-310650cd3a64 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:57.115912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 690/1377] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:57 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:57.117544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0914186-59e2-4bac-9b38-60bba51c8034 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:57.118111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0914186-59e2-4bac-9b38-60bba51c8034 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:57.118111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0914186-59e2-4bac-9b38-60bba51c8034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:57.138053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c0914186-59e2-4bac-9b38-60bba51c8034 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:57.139047 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c0914186-59e2-4bac-9b38-60bba51c8034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:57.156988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 688/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:57.196636 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2fb43-bc6b-42cb-8838-310650cd3a64 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:57.213700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2fb43-bc6b-42cb-8838-310650cd3a64 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:57.219847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc41d7c1-33a6-45d0-b3ea-1ca395953bc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:57.220694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41d7c1-33a6-45d0-b3ea-1ca395953bc3 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:57.221153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41d7c1-33a6-45d0-b3ea-1ca395953bc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:57.221571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc41d7c1-33a6-45d0-b3ea-1ca395953bc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:57.231002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41d7c1-33a6-45d0-b3ea-1ca395953bc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:57.232680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc41d7c1-33a6-45d0-b3ea-1ca395953bc3 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:57.236911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:26:57.251477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 691/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:26:59.331179 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d0d7-75d1-43d7-b51a-16e65348f962 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:59.331767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d0d7-75d1-43d7-b51a-16e65348f962 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:59.331767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d0d7-75d1-43d7-b51a-16e65348f962 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:59.333163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 689/1380] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:26:59.338292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020210df-16b2-4011-87d6-eab4ba95c24d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:59.338712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020210df-16b2-4011-87d6-eab4ba95c24d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:59.338849 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020210df-16b2-4011-87d6-eab4ba95c24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:59.356815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-020210df-16b2-4011-87d6-eab4ba95c24d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:26:59.358222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-020210df-16b2-4011-87d6-eab4ba95c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:26:59.367645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 692/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:26:59.420992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7d0d7-75d1-43d7-b51a-16e65348f962 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:26:59.436227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7d0d7-75d1-43d7-b51a-16e65348f962 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:26:59.441919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb5371-5957-45fa-9bd6-8b321eff5799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:26:59.442564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb5371-5957-45fa-9bd6-8b321eff5799 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:26:59.442850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb5371-5957-45fa-9bd6-8b321eff5799 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:26:59.443127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb5371-5957-45fa-9bd6-8b321eff5799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:26:59.451505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb5371-5957-45fa-9bd6-8b321eff5799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:26:59.453107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb5371-5957-45fa-9bd6-8b321eff5799 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:26:59.462732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 690/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:01.582404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8399084-1e66-43d3-859a-c074242b346f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:01.583286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8399084-1e66-43d3-859a-c074242b346f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:01.583779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8399084-1e66-43d3-859a-c074242b346f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:01.585689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 693/1383] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:27:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:01.590878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80014763-4521-40b4-8db7-05d2565f538d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:01.591527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80014763-4521-40b4-8db7-05d2565f538d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:01.591527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80014763-4521-40b4-8db7-05d2565f538d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:01.611163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-80014763-4521-40b4-8db7-05d2565f538d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:01.612629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-80014763-4521-40b4-8db7-05d2565f538d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:01.622083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 691/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:01.664436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8399084-1e66-43d3-859a-c074242b346f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:01.680640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8399084-1e66-43d3-859a-c074242b346f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:01.686172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249614f-7a09-4238-86af-488b35f941ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:01.687066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249614f-7a09-4238-86af-488b35f941ae service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:01.687557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249614f-7a09-4238-86af-488b35f941ae service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:01.688015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249614f-7a09-4238-86af-488b35f941ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:01.697173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249614f-7a09-4238-86af-488b35f941ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:01.698891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249614f-7a09-4238-86af-488b35f941ae service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:01.709135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 694/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:03.774432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fba6ae-65df-48c0-982e-0658f36b7cfe None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:03.774868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fba6ae-65df-48c0-982e-0658f36b7cfe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:03.775417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fba6ae-65df-48c0-982e-0658f36b7cfe None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:03.777947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 692/1386] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:27:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:03.783578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341f2d9-b2c3-4244-99c0-4924570b88e5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:03.784189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341f2d9-b2c3-4244-99c0-4924570b88e5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:03.784189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341f2d9-b2c3-4244-99c0-4924570b88e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:03.808624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e341f2d9-b2c3-4244-99c0-4924570b88e5 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:03.809925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e341f2d9-b2c3-4244-99c0-4924570b88e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:03.818279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 695/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:03.862486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fba6ae-65df-48c0-982e-0658f36b7cfe None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:03.877484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fba6ae-65df-48c0-982e-0658f36b7cfe None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:03.882903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e2e98-0619-4327-a260-453e7a8635ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:03.883603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e2e98-0619-4327-a260-453e7a8635ee service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:03.883890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e2e98-0619-4327-a260-453e7a8635ee service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:03.884184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e2e98-0619-4327-a260-453e7a8635ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:03.894914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e2e98-0619-4327-a260-453e7a8635ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:03.896585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e2e98-0619-4327-a260-453e7a8635ee service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:03.907383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 693/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:06.241930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad7a56-2842-4203-9684-41eb10de271a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:06.242476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad7a56-2842-4203-9684-41eb10de271a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:06.242476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad7a56-2842-4203-9684-41eb10de271a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:06.243973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 696/1389] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:06.250651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e60a6e-d197-4115-9195-3e07fb500fe7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:06.251128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e60a6e-d197-4115-9195-3e07fb500fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:06.251327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e60a6e-d197-4115-9195-3e07fb500fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:06.273587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-37e60a6e-d197-4115-9195-3e07fb500fe7 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:06.275419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-37e60a6e-d197-4115-9195-3e07fb500fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:06.285590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 694/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:27:06.339285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad7a56-2842-4203-9684-41eb10de271a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:06.354563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad7a56-2842-4203-9684-41eb10de271a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:06.360314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604c3d3-8a80-4a72-967f-2d01784bc983 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:06.361014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604c3d3-8a80-4a72-967f-2d01784bc983 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:06.361230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604c3d3-8a80-4a72-967f-2d01784bc983 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:06.361942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604c3d3-8a80-4a72-967f-2d01784bc983 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:06.370472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604c3d3-8a80-4a72-967f-2d01784bc983 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:06.372109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604c3d3-8a80-4a72-967f-2d01784bc983 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:06.382004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 697/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:08.914006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3041f6-d72b-44e8-8f4e-096338b94ea4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:08.914637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3041f6-d72b-44e8-8f4e-096338b94ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:08.914637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3041f6-d72b-44e8-8f4e-096338b94ea4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:08.916200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 695/1392] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:27:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:08.922435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb882e49-8f8e-46a7-b1f0-a9af4017dddb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:08.922877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb882e49-8f8e-46a7-b1f0-a9af4017dddb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:08.922971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb882e49-8f8e-46a7-b1f0-a9af4017dddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:08.943829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bb882e49-8f8e-46a7-b1f0-a9af4017dddb None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:08.945312 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bb882e49-8f8e-46a7-b1f0-a9af4017dddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:08.955623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 698/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:09.003538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3041f6-d72b-44e8-8f4e-096338b94ea4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:09.019003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3041f6-d72b-44e8-8f4e-096338b94ea4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:09.024650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d416bc-6ae9-4f07-bc90-a98b38678606 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:09.025334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d416bc-6ae9-4f07-bc90-a98b38678606 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:09.025620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d416bc-6ae9-4f07-bc90-a98b38678606 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:09.025898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d416bc-6ae9-4f07-bc90-a98b38678606 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:09.034727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d416bc-6ae9-4f07-bc90-a98b38678606 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:09.036399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d416bc-6ae9-4f07-bc90-a98b38678606 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:09.046251 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 696/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:11.125668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755369-60f8-49c6-b4df-22af3182e874 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:11.126174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755369-60f8-49c6-b4df-22af3182e874 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:11.129900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755369-60f8-49c6-b4df-22af3182e874 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:11.131550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 699/1395] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:27:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:11.137362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59573b89-f48e-407d-adea-4fd4667b76e2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:11.137831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59573b89-f48e-407d-adea-4fd4667b76e2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:11.137949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59573b89-f48e-407d-adea-4fd4667b76e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:11.156574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-59573b89-f48e-407d-adea-4fd4667b76e2 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:11.157779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-59573b89-f48e-407d-adea-4fd4667b76e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:11.166411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 697/1396] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:27:11.211155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755369-60f8-49c6-b4df-22af3182e874 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:11.226320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755369-60f8-49c6-b4df-22af3182e874 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:11.232419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82186dc-f368-4cda-abea-91e808780601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:11.233071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82186dc-f368-4cda-abea-91e808780601 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:11.233349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82186dc-f368-4cda-abea-91e808780601 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:11.233619 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82186dc-f368-4cda-abea-91e808780601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:11.242428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82186dc-f368-4cda-abea-91e808780601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:11.244159 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82186dc-f368-4cda-abea-91e808780601 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:11.254603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 700/1397] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:11.408710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bbcf4-91cc-421b-a05c-f34e7482163d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:11.408950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bbcf4-91cc-421b-a05c-f34e7482163d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:11.409262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bbcf4-91cc-421b-a05c-f34e7482163d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:11.410778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 698/1398] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:11.417174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d047d0-c281-4c1d-9738-a27154c45836 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:11.417567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d047d0-c281-4c1d-9738-a27154c45836 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:11.417860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d047d0-c281-4c1d-9738-a27154c45836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:11.437251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-29d047d0-c281-4c1d-9738-a27154c45836 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:11.438565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-29d047d0-c281-4c1d-9738-a27154c45836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:11.448522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 701/1399] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:11.503933 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc4c8f-739e-4b55-9cb0-456923c1bb8c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:11.504221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc4c8f-739e-4b55-9cb0-456923c1bb8c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:11.504496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc4c8f-739e-4b55-9cb0-456923c1bb8c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:11.505980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 699/1400] 10.222.0.42 () {54 vars in 962 bytes} [Sat Jul 2 03:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:11.511261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6858f25-ef69-4367-9c41-dfcc67480e8a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:11.511587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6858f25-ef69-4367-9c41-dfcc67480e8a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:11.511867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6858f25-ef69-4367-9c41-dfcc67480e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:11.561332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d6858f25-ef69-4367-9c41-dfcc67480e8a None None] MFA Rules not processed for user `2afd25850dea40568271538793e861c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:11.562532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d6858f25-ef69-4367-9c41-dfcc67480e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:11.624352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 702/1401] 10.222.0.42 () {58 vars in 1059 bytes} [Sat Jul 2 03:27:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:11.632278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc4c8f-739e-4b55-9cb0-456923c1bb8c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:11.644415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc4c8f-739e-4b55-9cb0-456923c1bb8c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:11.648414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a613da-e21a-4be3-8b24-883b56e9c45c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:11.649117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a613da-e21a-4be3-8b24-883b56e9c45c service glance] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:11.649399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a613da-e21a-4be3-8b24-883b56e9c45c service glance] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:11.649674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a613da-e21a-4be3-8b24-883b56e9c45c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:11.659711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a613da-e21a-4be3-8b24-883b56e9c45c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:11.661269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a613da-e21a-4be3-8b24-883b56e9c45c service glance] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:11.670945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 700/1402] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:12.418507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75942d57-6a67-474a-98ea-082820958b9a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:12.419060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75942d57-6a67-474a-98ea-082820958b9a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:12.419060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75942d57-6a67-474a-98ea-082820958b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:12.467176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-75942d57-6a67-474a-98ea-082820958b9a None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:12.468508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-75942d57-6a67-474a-98ea-082820958b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:12.509010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:27:12.534228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 703/1403] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:27:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:12.574908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc72fb-b437-4ce9-bb5f-6ae0081d8e7c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:12.575463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc72fb-b437-4ce9-bb5f-6ae0081d8e7c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:12.575679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc72fb-b437-4ce9-bb5f-6ae0081d8e7c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:12.577183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 701/1404] 10.222.0.42 () {54 vars in 942 bytes} [Sat Jul 2 03:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:12.583829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48bd52-16ca-4388-b2fd-ed1abe32c70c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:12.584062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48bd52-16ca-4388-b2fd-ed1abe32c70c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:12.584338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48bd52-16ca-4388-b2fd-ed1abe32c70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:12.633166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eb48bd52-16ca-4388-b2fd-ed1abe32c70c None None] MFA Rules not processed for user `f37a95ba40774cfd89e6f397afa301a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:12.634545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eb48bd52-16ca-4388-b2fd-ed1abe32c70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:12.717787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 704/1405] 10.222.0.42 () {58 vars in 1039 bytes} [Sat Jul 2 03:27:12 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:12.727600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc72fb-b437-4ce9-bb5f-6ae0081d8e7c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:12.736056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=61 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:27:12.745197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc72fb-b437-4ce9-bb5f-6ae0081d8e7c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:12.752801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aff59-07d4-4762-a5be-b8a1cc77d0b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:12.753916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aff59-07d4-4762-a5be-b8a1cc77d0b6 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:12.754598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aff59-07d4-4762-a5be-b8a1cc77d0b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:12.754896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aff59-07d4-4762-a5be-b8a1cc77d0b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:12.765537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aff59-07d4-4762-a5be-b8a1cc77d0b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:12.767170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3aff59-07d4-4762-a5be-b8a1cc77d0b6 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:12.775871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 702/1406] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:27:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:27:13.346704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7e03-1cba-4493-b9b2-ef484c96384a None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:13.347001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7e03-1cba-4493-b9b2-ef484c96384a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:13.347284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7e03-1cba-4493-b9b2-ef484c96384a None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:13.351782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 705/1407] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:27:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:13.354441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb66f93-6f90-462c-b8bd-44682850a040 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:13.354735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb66f93-6f90-462c-b8bd-44682850a040 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:13.355042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb66f93-6f90-462c-b8bd-44682850a040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:13.374454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4bb66f93-6f90-462c-b8bd-44682850a040 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:13.375613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4bb66f93-6f90-462c-b8bd-44682850a040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:13.388551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 703/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:13.437297 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf7e03-1cba-4493-b9b2-ef484c96384a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:13.453079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf7e03-1cba-4493-b9b2-ef484c96384a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:13.459318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e13db5-15c2-41d1-92da-da4f07120b55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:13.461970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e13db5-15c2-41d1-92da-da4f07120b55 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:13.461970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e13db5-15c2-41d1-92da-da4f07120b55 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:13.461970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e13db5-15c2-41d1-92da-da4f07120b55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:13.471370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e13db5-15c2-41d1-92da-da4f07120b55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:13.473394 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e13db5-15c2-41d1-92da-da4f07120b55 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:13.484130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 706/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:23.369428 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=8 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:27:23.479158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=59 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:27:34.883329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cf1c4-9751-48fe-bab8-83efae24ed05 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:34.883932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cf1c4-9751-48fe-bab8-83efae24ed05 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:34.883932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cf1c4-9751-48fe-bab8-83efae24ed05 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:34.885346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 704/1410] 10.222.0.42 () {54 vars in 918 bytes} [Sat Jul 2 03:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:34.890540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af2f0-f2f8-42e3-bac8-974ba17fa80e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:34.890988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af2f0-f2f8-42e3-bac8-974ba17fa80e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:34.891091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af2f0-f2f8-42e3-bac8-974ba17fa80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:34.910931 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-285af2f0-f2f8-42e3-bac8-974ba17fa80e None None] MFA Rules not processed for user `6ee7acbd4c5545c1885faffb71f6306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:34.912183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-285af2f0-f2f8-42e3-bac8-974ba17fa80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:34.921359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 707/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Jul 2 03:27:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:34.936956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0610ee8-299c-4146-8593-c15f12f639b3 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:34.937330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0610ee8-299c-4146-8593-c15f12f639b3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:34.937737 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0610ee8-299c-4146-8593-c15f12f639b3 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:34.940301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 705/1412] 10.222.0.42 () {54 vars in 918 bytes} [Sat Jul 2 03:27:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:34.945295 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2893e0-cf23-48ea-a554-00d312ea07d4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:34.945586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2893e0-cf23-48ea-a554-00d312ea07d4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:34.945869 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2893e0-cf23-48ea-a554-00d312ea07d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:34.964669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2f2893e0-cf23-48ea-a554-00d312ea07d4 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:34.965863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2f2893e0-cf23-48ea-a554-00d312ea07d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:34.977823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 708/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Sat Jul 2 03:27:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:37.458729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2dad3-d1e0-4e71-ad33-3820c37b2a3f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:37.459622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2dad3-d1e0-4e71-ad33-3820c37b2a3f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:37.459622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2dad3-d1e0-4e71-ad33-3820c37b2a3f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:37.461792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:27:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:37.467213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974fe68-1b3d-4ea1-8d8c-83ee8d873ccd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:37.467691 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974fe68-1b3d-4ea1-8d8c-83ee8d873ccd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:37.467815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974fe68-1b3d-4ea1-8d8c-83ee8d873ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:37.486057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3974fe68-1b3d-4ea1-8d8c-83ee8d873ccd None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:37.487201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3974fe68-1b3d-4ea1-8d8c-83ee8d873ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:37.495548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:37.541596 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2dad3-d1e0-4e71-ad33-3820c37b2a3f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:37.566313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2dad3-d1e0-4e71-ad33-3820c37b2a3f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:37.572874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c800882-5136-4739-abc2-db09bd35302a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:37.573551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800882-5136-4739-abc2-db09bd35302a service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:37.573837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800882-5136-4739-abc2-db09bd35302a service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:37.574118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800882-5136-4739-abc2-db09bd35302a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:37.582758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c800882-5136-4739-abc2-db09bd35302a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:37.584397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c800882-5136-4739-abc2-db09bd35302a service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:37.593937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 707/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:38.499240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54722c23-07d9-47fc-8938-a5a4a464d1c4 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:38.499828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54722c23-07d9-47fc-8938-a5a4a464d1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:38.499828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54722c23-07d9-47fc-8938-a5a4a464d1c4 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:38.501271 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:38.507354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1419c7-1705-4445-a020-c9205b2e2ac7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:38.508094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1419c7-1705-4445-a020-c9205b2e2ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:38.508304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1419c7-1705-4445-a020-c9205b2e2ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:38.531096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cc1419c7-1705-4445-a020-c9205b2e2ac7 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:38.532686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cc1419c7-1705-4445-a020-c9205b2e2ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:38.541563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:38.709492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54722c23-07d9-47fc-8938-a5a4a464d1c4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:38.729765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54722c23-07d9-47fc-8938-a5a4a464d1c4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:38.738532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afc7ef-0221-4be3-92bb-599068e7e999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:38.739178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afc7ef-0221-4be3-92bb-599068e7e999 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:38.739496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afc7ef-0221-4be3-92bb-599068e7e999 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:38.739771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afc7ef-0221-4be3-92bb-599068e7e999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:38.748846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afc7ef-0221-4be3-92bb-599068e7e999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:38.750564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afc7ef-0221-4be3-92bb-599068e7e999 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:38.760887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 711/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:39.660383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64c41a-b354-4fd1-b582-24b0f2a74c8a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:39.660950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64c41a-b354-4fd1-b582-24b0f2a74c8a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:39.660950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64c41a-b354-4fd1-b582-24b0f2a74c8a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:39.662394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:39.668441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f99c8a-dbb8-4941-854f-7b3113162b1b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:39.669222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f99c8a-dbb8-4941-854f-7b3113162b1b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:39.669401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f99c8a-dbb8-4941-854f-7b3113162b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:39.692380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c5f99c8a-dbb8-4941-854f-7b3113162b1b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:39.693588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c5f99c8a-dbb8-4941-854f-7b3113162b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:39.702711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:39.856818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64c41a-b354-4fd1-b582-24b0f2a74c8a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:39.879467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64c41a-b354-4fd1-b582-24b0f2a74c8a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:39.885199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c49ac-bfab-4aed-a0c3-edf3ab8c16dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:39.885841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c49ac-bfab-4aed-a0c3-edf3ab8c16dc service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:39.886119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c49ac-bfab-4aed-a0c3-edf3ab8c16dc service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:39.886395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c49ac-bfab-4aed-a0c3-edf3ab8c16dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:39.895545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c49ac-bfab-4aed-a0c3-edf3ab8c16dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:39.897131 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c49ac-bfab-4aed-a0c3-edf3ab8c16dc service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:39.906802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 710/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:40.797595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f58932-9e1d-45a6-9af6-20b9c6859997 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:40.798205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f58932-9e1d-45a6-9af6-20b9c6859997 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:40.798205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f58932-9e1d-45a6-9af6-20b9c6859997 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:40.799609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:27:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:40.805156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dd747-0a4d-4516-8b6e-ef247016368e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:40.805590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dd747-0a4d-4516-8b6e-ef247016368e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:40.805705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dd747-0a4d-4516-8b6e-ef247016368e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:40.825479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6c2dd747-0a4d-4516-8b6e-ef247016368e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:40.826648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6c2dd747-0a4d-4516-8b6e-ef247016368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:40.835382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:40.988934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f58932-9e1d-45a6-9af6-20b9c6859997 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:41.014234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f58932-9e1d-45a6-9af6-20b9c6859997 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:41.020909 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0540734-49d8-42b4-ac24-3a3b39250110 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:41.021542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0540734-49d8-42b4-ac24-3a3b39250110 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:41.022251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0540734-49d8-42b4-ac24-3a3b39250110 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:41.022251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0540734-49d8-42b4-ac24-3a3b39250110 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:41.030964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0540734-49d8-42b4-ac24-3a3b39250110 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:41.032620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0540734-49d8-42b4-ac24-3a3b39250110 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:41.041971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 714/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:41.908761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d3b69-93c2-494a-b75c-e0d3d00108a4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:41.909324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d3b69-93c2-494a-b75c-e0d3d00108a4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:41.909324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d3b69-93c2-494a-b75c-e0d3d00108a4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:41.911344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:27:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:41.919264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc3159-58e5-4d65-9ef0-c740f3bc880c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:41.919721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc3159-58e5-4d65-9ef0-c740f3bc880c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:41.919883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc3159-58e5-4d65-9ef0-c740f3bc880c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:41.942705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eddc3159-58e5-4d65-9ef0-c740f3bc880c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:41.943913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eddc3159-58e5-4d65-9ef0-c740f3bc880c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:41.952366 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:42.102549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d3b69-93c2-494a-b75c-e0d3d00108a4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:42.125485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d3b69-93c2-494a-b75c-e0d3d00108a4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:42.131762 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca603987-46e6-49e5-bd41-367c3664cca1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:42.132415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca603987-46e6-49e5-bd41-367c3664cca1 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:42.132695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca603987-46e6-49e5-bd41-367c3664cca1 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:42.132967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca603987-46e6-49e5-bd41-367c3664cca1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:42.142517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca603987-46e6-49e5-bd41-367c3664cca1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:42.144161 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca603987-46e6-49e5-bd41-367c3664cca1 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:42.154525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 713/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:42.959562 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cea688-1660-43b5-9039-af457aa0b812 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:42.960498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cea688-1660-43b5-9039-af457aa0b812 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:42.960498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cea688-1660-43b5-9039-af457aa0b812 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:42.962685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:27:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:42.968859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73273c9-8e05-4f26-b77d-b1a70b367d2b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:42.969497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73273c9-8e05-4f26-b77d-b1a70b367d2b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:42.969497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73273c9-8e05-4f26-b77d-b1a70b367d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:42.994340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e73273c9-8e05-4f26-b77d-b1a70b367d2b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:42.995602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e73273c9-8e05-4f26-b77d-b1a70b367d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:43.005701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:43.155790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cea688-1660-43b5-9039-af457aa0b812 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:43.172843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cea688-1660-43b5-9039-af457aa0b812 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:43.179023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111c08a-79c4-4960-8f3c-5bcb5861c756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:43.179693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111c08a-79c4-4960-8f3c-5bcb5861c756 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:43.179974 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111c08a-79c4-4960-8f3c-5bcb5861c756 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:43.180248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111c08a-79c4-4960-8f3c-5bcb5861c756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:43.188919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111c08a-79c4-4960-8f3c-5bcb5861c756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:43.190855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111c08a-79c4-4960-8f3c-5bcb5861c756 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:43.200491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 717/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:44.005622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d3971-509d-48fb-b628-203fd07fc9e7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:44.006577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d3971-509d-48fb-b628-203fd07fc9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:44.006577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d3971-509d-48fb-b628-203fd07fc9e7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:44.008922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:27:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:44.017742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ee2fd-9d88-4c7a-95b8-fc40f2a1ebc3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:44.018207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ee2fd-9d88-4c7a-95b8-fc40f2a1ebc3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:44.018318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ee2fd-9d88-4c7a-95b8-fc40f2a1ebc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:44.039026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-801ee2fd-9d88-4c7a-95b8-fc40f2a1ebc3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:44.040305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-801ee2fd-9d88-4c7a-95b8-fc40f2a1ebc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:44.050176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:44.205565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d3971-509d-48fb-b628-203fd07fc9e7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:44.227958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d3971-509d-48fb-b628-203fd07fc9e7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:44.235705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdfabc-a017-448b-9882-7e6fca41ad3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:44.236660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdfabc-a017-448b-9882-7e6fca41ad3b service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:44.236960 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdfabc-a017-448b-9882-7e6fca41ad3b service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:44.237353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdfabc-a017-448b-9882-7e6fca41ad3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:44.251257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdfabc-a017-448b-9882-7e6fca41ad3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:44.252894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdfabc-a017-448b-9882-7e6fca41ad3b service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:44.262890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 716/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:45.092344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9e2ed-4bc8-4cbc-8efc-8fcb77556461 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:45.093162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9e2ed-4bc8-4cbc-8efc-8fcb77556461 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:45.093162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9e2ed-4bc8-4cbc-8efc-8fcb77556461 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:45.094893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:27:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:45.100074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a97ef-db93-4450-9380-5c47b8d4b5fc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:45.100499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a97ef-db93-4450-9380-5c47b8d4b5fc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:45.100620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a97ef-db93-4450-9380-5c47b8d4b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:45.119215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-817a97ef-db93-4450-9380-5c47b8d4b5fc None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:45.120420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-817a97ef-db93-4450-9380-5c47b8d4b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:45.130553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:27:45.295649 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9e2ed-4bc8-4cbc-8efc-8fcb77556461 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:45.312360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9e2ed-4bc8-4cbc-8efc-8fcb77556461 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:45.318992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5fd16-556a-4a69-adef-e56f3b189119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:45.319667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5fd16-556a-4a69-adef-e56f3b189119 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:45.319947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5fd16-556a-4a69-adef-e56f3b189119 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:45.320214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5fd16-556a-4a69-adef-e56f3b189119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:45.329275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5fd16-556a-4a69-adef-e56f3b189119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:45.330904 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5fd16-556a-4a69-adef-e56f3b189119 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:45.340842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 720/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:46.138969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b2d3a-0a70-4230-816e-7aa9ad9fae1d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:46.139842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b2d3a-0a70-4230-816e-7aa9ad9fae1d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:46.139842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b2d3a-0a70-4230-816e-7aa9ad9fae1d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:46.141733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:27:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:46.146840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850329-fef3-49e4-bb72-30c7d0e9086b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:46.147285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850329-fef3-49e4-bb72-30c7d0e9086b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:46.147434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9850329-fef3-49e4-bb72-30c7d0e9086b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:46.166418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e9850329-fef3-49e4-bb72-30c7d0e9086b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:46.167657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e9850329-fef3-49e4-bb72-30c7d0e9086b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:46.177385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:46.328470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b2d3a-0a70-4230-816e-7aa9ad9fae1d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:46.348116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b2d3a-0a70-4230-816e-7aa9ad9fae1d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:46.354810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5671e93-c944-4591-ae7e-0494c1d76f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:46.355482 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5671e93-c944-4591-ae7e-0494c1d76f97 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:46.355768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5671e93-c944-4591-ae7e-0494c1d76f97 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:46.356049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5671e93-c944-4591-ae7e-0494c1d76f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:46.365570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5671e93-c944-4591-ae7e-0494c1d76f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:46.367106 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5671e93-c944-4591-ae7e-0494c1d76f97 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:46.377243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 719/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:48.291419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879b13-22cd-46e2-a1a1-8644a79375a2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:48.292264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879b13-22cd-46e2-a1a1-8644a79375a2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:48.292264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d879b13-22cd-46e2-a1a1-8644a79375a2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:48.294339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:27:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:48.300423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd53d43-2054-44dd-9044-d6d51e6760a2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:48.300890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd53d43-2054-44dd-9044-d6d51e6760a2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:48.300992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd53d43-2054-44dd-9044-d6d51e6760a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:48.328652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5cd53d43-2054-44dd-9044-d6d51e6760a2 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:48.329876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5cd53d43-2054-44dd-9044-d6d51e6760a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:48.338994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:48.377034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879b13-22cd-46e2-a1a1-8644a79375a2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:48.390861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d879b13-22cd-46e2-a1a1-8644a79375a2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:48.395388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca592e-b30b-4e40-aabc-d09b67a9282e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:48.396045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca592e-b30b-4e40-aabc-d09b67a9282e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:48.396322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca592e-b30b-4e40-aabc-d09b67a9282e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:48.396322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca592e-b30b-4e40-aabc-d09b67a9282e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:48.408901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca592e-b30b-4e40-aabc-d09b67a9282e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:48.410402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca592e-b30b-4e40-aabc-d09b67a9282e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:48.420901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 723/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:50.637097 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779444bc-2b6f-47f9-a3a3-54c58b0192d0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:50.638019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779444bc-2b6f-47f9-a3a3-54c58b0192d0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:50.638181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779444bc-2b6f-47f9-a3a3-54c58b0192d0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:50.640257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:27:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:50.648671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fadb9a-6606-4d03-a060-2014e98d97b9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:50.649995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fadb9a-6606-4d03-a060-2014e98d97b9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:50.650279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fadb9a-6606-4d03-a060-2014e98d97b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:50.678684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-93fadb9a-6606-4d03-a060-2014e98d97b9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:50.679917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-93fadb9a-6606-4d03-a060-2014e98d97b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:50.689138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:27:50.736001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779444bc-2b6f-47f9-a3a3-54c58b0192d0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:50.755207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779444bc-2b6f-47f9-a3a3-54c58b0192d0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:50.761660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96931986-5f2b-4681-a044-64aa273ca17e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:50.762791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96931986-5f2b-4681-a044-64aa273ca17e service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:50.763359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96931986-5f2b-4681-a044-64aa273ca17e service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:50.763879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96931986-5f2b-4681-a044-64aa273ca17e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:50.775792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96931986-5f2b-4681-a044-64aa273ca17e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:50.777274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96931986-5f2b-4681-a044-64aa273ca17e service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:50.789569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 722/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:53.211806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184af766-46b4-4e9c-bea7-d90ec764b04c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:53.212358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184af766-46b4-4e9c-bea7-d90ec764b04c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:53.212358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184af766-46b4-4e9c-bea7-d90ec764b04c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:53.213766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:53.219227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291619d-4c7b-46e7-a19f-3c615b028171 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:53.219711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291619d-4c7b-46e7-a19f-3c615b028171 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:53.219852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291619d-4c7b-46e7-a19f-3c615b028171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:53.239675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f291619d-4c7b-46e7-a19f-3c615b028171 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:53.240868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f291619d-4c7b-46e7-a19f-3c615b028171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:53.250571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:27:53.292677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184af766-46b4-4e9c-bea7-d90ec764b04c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:53.310081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184af766-46b4-4e9c-bea7-d90ec764b04c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:53.315038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d27d-16d2-4a74-b8ac-b149177b2904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:53.315720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d27d-16d2-4a74-b8ac-b149177b2904 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:53.316003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d27d-16d2-4a74-b8ac-b149177b2904 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:53.316275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d27d-16d2-4a74-b8ac-b149177b2904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:53.326425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4d27d-16d2-4a74-b8ac-b149177b2904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:53.327930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4d27d-16d2-4a74-b8ac-b149177b2904 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:53.338588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 726/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:55.606124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb0e59-c6fe-4b21-af84-52bd5828ab6f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:55.607175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb0e59-c6fe-4b21-af84-52bd5828ab6f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:55.607175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb0e59-c6fe-4b21-af84-52bd5828ab6f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:55.609406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:27:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:55.616394 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb42bb8-1316-4f05-a728-a25c5413cac9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:55.616885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb42bb8-1316-4f05-a728-a25c5413cac9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:55.616992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb42bb8-1316-4f05-a728-a25c5413cac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:55.639627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2cb42bb8-1316-4f05-a728-a25c5413cac9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:55.640911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2cb42bb8-1316-4f05-a728-a25c5413cac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:55.651269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:27:55.691321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb0e59-c6fe-4b21-af84-52bd5828ab6f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:55.703910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb0e59-c6fe-4b21-af84-52bd5828ab6f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:55.707748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871eea2-2329-4065-bf92-d5e5839828ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:55.708386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871eea2-2329-4065-bf92-d5e5839828ed service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:55.708650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871eea2-2329-4065-bf92-d5e5839828ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:55.708932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871eea2-2329-4065-bf92-d5e5839828ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:55.718202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871eea2-2329-4065-bf92-d5e5839828ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:55.719703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871eea2-2329-4065-bf92-d5e5839828ed service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:55.729250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 725/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:27:57.804843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc92f3-c760-4140-8ccf-097a0b2860ba None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:57.805759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc92f3-c760-4140-8ccf-097a0b2860ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:57.805759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc92f3-c760-4140-8ccf-097a0b2860ba None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:57.808044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:27:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:27:57.815549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39abc99-3f2e-4bb6-8ae0-a47a9230b50e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:57.815990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39abc99-3f2e-4bb6-8ae0-a47a9230b50e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:57.816111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39abc99-3f2e-4bb6-8ae0-a47a9230b50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:57.837977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d39abc99-3f2e-4bb6-8ae0-a47a9230b50e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:27:57.839189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d39abc99-3f2e-4bb6-8ae0-a47a9230b50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:27:57.849153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:27:57.890489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc92f3-c760-4140-8ccf-097a0b2860ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:27:57.907215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc92f3-c760-4140-8ccf-097a0b2860ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:27:57.910851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3bc345-1e59-4aff-81ba-769869da5c4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:27:57.911558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bc345-1e59-4aff-81ba-769869da5c4e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:27:57.911843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bc345-1e59-4aff-81ba-769869da5c4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:27:57.912120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3bc345-1e59-4aff-81ba-769869da5c4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:27:57.921191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bc345-1e59-4aff-81ba-769869da5c4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:27:57.922782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3bc345-1e59-4aff-81ba-769869da5c4e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:27:57.932740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 729/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:28:00.036138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da531a3-9caf-4ace-8680-a93b28a194ab None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:00.036815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da531a3-9caf-4ace-8680-a93b28a194ab None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:00.036815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da531a3-9caf-4ace-8680-a93b28a194ab None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:00.038128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:00.044684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b7011-125d-4722-99c9-58d817d688ec None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:00.045440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b7011-125d-4722-99c9-58d817d688ec None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:00.045586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b7011-125d-4722-99c9-58d817d688ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:00.070759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ca5b7011-125d-4722-99c9-58d817d688ec None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:00.071991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ca5b7011-125d-4722-99c9-58d817d688ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:00.080825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:00.119502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da531a3-9caf-4ace-8680-a93b28a194ab None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:00.131941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da531a3-9caf-4ace-8680-a93b28a194ab None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:00.136211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8476e6c-f672-4554-84e9-c18a45c400c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:00.136862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476e6c-f672-4554-84e9-c18a45c400c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:00.137145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476e6c-f672-4554-84e9-c18a45c400c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:00.137405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8476e6c-f672-4554-84e9-c18a45c400c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:00.147308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476e6c-f672-4554-84e9-c18a45c400c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:00.148766 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8476e6c-f672-4554-84e9-c18a45c400c9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:00.158597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 728/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:28:02.312464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f9fd-baca-4992-9859-53d87db4b719 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:02.313064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f9fd-baca-4992-9859-53d87db4b719 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:02.313064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148f9fd-baca-4992-9859-53d87db4b719 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:02.314468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:02.320963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0683a2-9058-4844-9bcf-292b0d1dbd7d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:02.321726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0683a2-9058-4844-9bcf-292b0d1dbd7d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:02.321898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0683a2-9058-4844-9bcf-292b0d1dbd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:02.345765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1f0683a2-9058-4844-9bcf-292b0d1dbd7d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:02.346963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1f0683a2-9058-4844-9bcf-292b0d1dbd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:02.355470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:02.396390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f9fd-baca-4992-9859-53d87db4b719 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:02.411877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148f9fd-baca-4992-9859-53d87db4b719 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:02.416124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c48942-9d82-401a-8e63-c6c01f9c6799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:02.416787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c48942-9d82-401a-8e63-c6c01f9c6799 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:02.417072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c48942-9d82-401a-8e63-c6c01f9c6799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:02.417372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c48942-9d82-401a-8e63-c6c01f9c6799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:02.427372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c48942-9d82-401a-8e63-c6c01f9c6799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:02.428806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c48942-9d82-401a-8e63-c6c01f9c6799 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:02.438910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 732/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:28:04.500994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0438d32-17c7-4f2e-a92c-65b39a7467a3 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:04.501574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0438d32-17c7-4f2e-a92c-65b39a7467a3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:04.501574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0438d32-17c7-4f2e-a92c-65b39a7467a3 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:04.502936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:04.509309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b6504-68e0-47b0-aa56-f603137226d4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:04.510054 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b6504-68e0-47b0-aa56-f603137226d4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:04.510226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b6504-68e0-47b0-aa56-f603137226d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:04.519596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=79 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:28:04.534362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2a3b6504-68e0-47b0-aa56-f603137226d4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:04.535580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2a3b6504-68e0-47b0-aa56-f603137226d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:04.545567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:04.590280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0438d32-17c7-4f2e-a92c-65b39a7467a3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:04.595919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=73 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:28:04.602059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0438d32-17c7-4f2e-a92c-65b39a7467a3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:04.606417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1908271-3108-4cf6-8d34-edfd57866481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:04.607056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908271-3108-4cf6-8d34-edfd57866481 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:04.607389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908271-3108-4cf6-8d34-edfd57866481 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:04.607682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1908271-3108-4cf6-8d34-edfd57866481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:04.617643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1908271-3108-4cf6-8d34-edfd57866481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:04.619094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1908271-3108-4cf6-8d34-edfd57866481 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:04.629105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 731/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:28:06.759182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddc0e6-bf9a-4dfb-9ceb-534b16363d00 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:06.759826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddc0e6-bf9a-4dfb-9ceb-534b16363d00 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:06.759826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddc0e6-bf9a-4dfb-9ceb-534b16363d00 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:06.761183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:06.767637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c98bfb-6b58-44d4-9cc8-88504e8becf4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:06.768394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c98bfb-6b58-44d4-9cc8-88504e8becf4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:06.768545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c98bfb-6b58-44d4-9cc8-88504e8becf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:06.791815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-66c98bfb-6b58-44d4-9cc8-88504e8becf4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:06.793020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-66c98bfb-6b58-44d4-9cc8-88504e8becf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:06.802958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:06.844200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddc0e6-bf9a-4dfb-9ceb-534b16363d00 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:06.856838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddc0e6-bf9a-4dfb-9ceb-534b16363d00 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:06.861145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299ad5c-6b6a-4915-abaf-14a1c53b54cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:06.861779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299ad5c-6b6a-4915-abaf-14a1c53b54cf service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:06.862061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299ad5c-6b6a-4915-abaf-14a1c53b54cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:06.862341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299ad5c-6b6a-4915-abaf-14a1c53b54cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:06.873125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299ad5c-6b6a-4915-abaf-14a1c53b54cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:06.874860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f299ad5c-6b6a-4915-abaf-14a1c53b54cf service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:06.884630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 735/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:28:08.952933 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67127ea-4a23-4dab-a28b-913fb3be773e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:08.953508 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67127ea-4a23-4dab-a28b-913fb3be773e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:08.953508 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67127ea-4a23-4dab-a28b-913fb3be773e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:08.954893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:08.961369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924a03a-8979-4b53-9a57-256a79c193dd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:08.962060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924a03a-8979-4b53-9a57-256a79c193dd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:08.962248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924a03a-8979-4b53-9a57-256a79c193dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:08.983530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c924a03a-8979-4b53-9a57-256a79c193dd None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:08.984726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c924a03a-8979-4b53-9a57-256a79c193dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:08.998769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:09.057390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67127ea-4a23-4dab-a28b-913fb3be773e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:09.069123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67127ea-4a23-4dab-a28b-913fb3be773e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:09.072943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c025-2ce3-46cb-8d12-6fb7c879784d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:09.073580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c025-2ce3-46cb-8d12-6fb7c879784d service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:09.073860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c025-2ce3-46cb-8d12-6fb7c879784d service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:09.074138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c025-2ce3-46cb-8d12-6fb7c879784d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:09.083487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4c025-2ce3-46cb-8d12-6fb7c879784d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:09.084924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4c025-2ce3-46cb-8d12-6fb7c879784d service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:09.094536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 734/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:28:18.978038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=11 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:28:19.090191 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=14 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:28:28.916183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd470b3-9111-407c-b03a-5e37133a459d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:28.916803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd470b3-9111-407c-b03a-5e37133a459d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:28.916803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd470b3-9111-407c-b03a-5e37133a459d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:28.918128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 737/1471] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:28.927341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a28f3-70dd-45ae-a18f-2f6860c79e49 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:28.927984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a28f3-70dd-45ae-a18f-2f6860c79e49 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:28.928171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a28f3-70dd-45ae-a18f-2f6860c79e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:28.955601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-138a28f3-70dd-45ae-a18f-2f6860c79e49 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:28.956805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-138a28f3-70dd-45ae-a18f-2f6860c79e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:28.968718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 735/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:29.026354 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d3353-1e36-4373-af1b-89fac4378c4d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:29.026643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d3353-1e36-4373-af1b-89fac4378c4d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:29.026918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d3353-1e36-4373-af1b-89fac4378c4d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:29.028338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 738/1473] 10.222.0.42 () {54 vars in 962 bytes} [Sat Jul 2 03:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:29.033351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6d22-99d0-4bc5-8f13-3008d478ce59 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:29.033637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6d22-99d0-4bc5-8f13-3008d478ce59 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:29.033947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf6d22-99d0-4bc5-8f13-3008d478ce59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:29.052706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-61cf6d22-99d0-4bc5-8f13-3008d478ce59 None None] MFA Rules not processed for user `2afd25850dea40568271538793e861c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:29.053931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-61cf6d22-99d0-4bc5-8f13-3008d478ce59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:29.060861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 736/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Sat Jul 2 03:28:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:29.071764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d3353-1e36-4373-af1b-89fac4378c4d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:29.082734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d3353-1e36-4373-af1b-89fac4378c4d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:29.086381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b0c7d-7904-4b2f-85da-5b50b574e6ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:29.087011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b0c7d-7904-4b2f-85da-5b50b574e6ac service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:29.087317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b0c7d-7904-4b2f-85da-5b50b574e6ac service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:29.087596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b0c7d-7904-4b2f-85da-5b50b574e6ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:29.097164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b0c7d-7904-4b2f-85da-5b50b574e6ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:29.098741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b0c7d-7904-4b2f-85da-5b50b574e6ac service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:29.108574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 739/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:28:31.270469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62816d53-4916-49c9-8aef-67469e5a653b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:31.271093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62816d53-4916-49c9-8aef-67469e5a653b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:31.271093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62816d53-4916-49c9-8aef-67469e5a653b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:31.272505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 737/1476] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:31.278867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4778a4-06fd-4db7-917d-d2c6df082d8b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:31.279698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4778a4-06fd-4db7-917d-d2c6df082d8b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:31.279907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4778a4-06fd-4db7-917d-d2c6df082d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:31.306425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2f4778a4-06fd-4db7-917d-d2c6df082d8b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:31.307556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2f4778a4-06fd-4db7-917d-d2c6df082d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:31.317369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 740/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:31.350912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62816d53-4916-49c9-8aef-67469e5a653b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:31.364815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62816d53-4916-49c9-8aef-67469e5a653b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:31.369479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15e866-0130-4b78-9641-87e570488101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:31.370135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15e866-0130-4b78-9641-87e570488101 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:31.370413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15e866-0130-4b78-9641-87e570488101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:31.370688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15e866-0130-4b78-9641-87e570488101 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:31.371461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15e866-0130-4b78-9641-87e570488101 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:31.373755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-fa15e866-0130-4b78-9641-87e570488101 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:28:31.373755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jul 02 03:28:31.373755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: Jul 02 03:28:31.375325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15e866-0130-4b78-9641-87e570488101 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:31.395971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 738/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Sat Jul 2 03:28:31 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:28:33.445298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc09cb4-36e8-4798-9df5-c4cd5a3951c3 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:33.446172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc09cb4-36e8-4798-9df5-c4cd5a3951c3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:33.446172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc09cb4-36e8-4798-9df5-c4cd5a3951c3 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:33.448383 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 741/1479] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:28:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:33.456594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bba6d-54ba-4cf2-ba03-359b7e44439d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:33.457031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bba6d-54ba-4cf2-ba03-359b7e44439d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:33.457179 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bba6d-54ba-4cf2-ba03-359b7e44439d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:33.485487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-de5bba6d-54ba-4cf2-ba03-359b7e44439d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:33.486717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-de5bba6d-54ba-4cf2-ba03-359b7e44439d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:33.499202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 739/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:33.542560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc09cb4-36e8-4798-9df5-c4cd5a3951c3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:33.558388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc09cb4-36e8-4798-9df5-c4cd5a3951c3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:33.564039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d92f3-9f7a-478f-9ed1-2ead29d0b2e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:33.564684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d92f3-9f7a-478f-9ed1-2ead29d0b2e2 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:33.564968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d92f3-9f7a-478f-9ed1-2ead29d0b2e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:33.565242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d92f3-9f7a-478f-9ed1-2ead29d0b2e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:33.573472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d92f3-9f7a-478f-9ed1-2ead29d0b2e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:33.575057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d92f3-9f7a-478f-9ed1-2ead29d0b2e2 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:33.584510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 742/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:28:35.788317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baacd5-7757-4202-a3f1-4af3b292c311 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:35.789238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baacd5-7757-4202-a3f1-4af3b292c311 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:35.789238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baacd5-7757-4202-a3f1-4af3b292c311 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:35.791501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 740/1482] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:28:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:35.799393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39967-3401-44fd-9635-3686aa78672e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:35.799866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39967-3401-44fd-9635-3686aa78672e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:35.799985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef39967-3401-44fd-9635-3686aa78672e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:35.819558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fef39967-3401-44fd-9635-3686aa78672e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:35.820756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fef39967-3401-44fd-9635-3686aa78672e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:35.829322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 743/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:35.867819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baacd5-7757-4202-a3f1-4af3b292c311 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:35.879397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baacd5-7757-4202-a3f1-4af3b292c311 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:35.882919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23223462-9e33-48ee-8c28-7af616fce0e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:35.883583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23223462-9e33-48ee-8c28-7af616fce0e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:35.883865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23223462-9e33-48ee-8c28-7af616fce0e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:35.884140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23223462-9e33-48ee-8c28-7af616fce0e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:35.893517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23223462-9e33-48ee-8c28-7af616fce0e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:35.894957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23223462-9e33-48ee-8c28-7af616fce0e9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:35.904580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 741/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:28:37.977051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0903a6-d937-48ef-9057-0b2e638afefa None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:37.977890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0903a6-d937-48ef-9057-0b2e638afefa None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:37.977890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0903a6-d937-48ef-9057-0b2e638afefa None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:37.980114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 744/1485] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:28:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:37.985751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9648898-4588-47bb-98dd-a1077b0f873e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:37.986183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9648898-4588-47bb-98dd-a1077b0f873e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:37.986276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9648898-4588-47bb-98dd-a1077b0f873e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:38.005412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b9648898-4588-47bb-98dd-a1077b0f873e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:38.006638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b9648898-4588-47bb-98dd-a1077b0f873e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:38.016004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 742/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:38.064338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0903a6-d937-48ef-9057-0b2e638afefa None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:38.084007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0903a6-d937-48ef-9057-0b2e638afefa None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:38.091110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5a6ab-3a37-48f7-9af9-8d77697e2ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:38.092042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5a6ab-3a37-48f7-9af9-8d77697e2ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:38.092489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5a6ab-3a37-48f7-9af9-8d77697e2ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:38.092932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5a6ab-3a37-48f7-9af9-8d77697e2ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:38.103529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5a6ab-3a37-48f7-9af9-8d77697e2ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:38.105002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5a6ab-3a37-48f7-9af9-8d77697e2ec5 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:38.115106 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 745/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:28:40.285548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28339e-45d3-4474-af75-c4d21647de82 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:40.286170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28339e-45d3-4474-af75-c4d21647de82 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:40.286170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28339e-45d3-4474-af75-c4d21647de82 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:40.288187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 743/1488] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:28:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:40.296497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bed7b6-8bd0-49f2-bac2-365bb243d095 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:40.297188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bed7b6-8bd0-49f2-bac2-365bb243d095 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:40.297463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bed7b6-8bd0-49f2-bac2-365bb243d095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:40.326752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f6bed7b6-8bd0-49f2-bac2-365bb243d095 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:40.328261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f6bed7b6-8bd0-49f2-bac2-365bb243d095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:40.342225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 746/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:40.385416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28339e-45d3-4474-af75-c4d21647de82 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:40.405647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28339e-45d3-4474-af75-c4d21647de82 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:40.410772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c50266-e1d8-4bd2-9f72-3f7f2685c801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:40.411474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c50266-e1d8-4bd2-9f72-3f7f2685c801 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:40.411756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c50266-e1d8-4bd2-9f72-3f7f2685c801 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:40.412030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c50266-e1d8-4bd2-9f72-3f7f2685c801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:40.421693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c50266-e1d8-4bd2-9f72-3f7f2685c801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:40.423164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c50266-e1d8-4bd2-9f72-3f7f2685c801 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:40.433235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 744/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:28:42.646639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ed944-529d-4fc9-b04e-08b69167d6c0 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:42.648136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ed944-529d-4fc9-b04e-08b69167d6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:42.648526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ed944-529d-4fc9-b04e-08b69167d6c0 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:42.649973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 747/1491] 10.222.0.42 () {54 vars in 926 bytes} [Sat Jul 2 03:28:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:28:42.655524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c843ac-57cc-4381-b0df-1b5815732e0b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:42.655979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c843ac-57cc-4381-b0df-1b5815732e0b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:42.656072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c843ac-57cc-4381-b0df-1b5815732e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:42.674504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-90c843ac-57cc-4381-b0df-1b5815732e0b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:28:42.676198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-90c843ac-57cc-4381-b0df-1b5815732e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:28:42.684743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 745/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Sat Jul 2 03:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:28:42.723725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ed944-529d-4fc9-b04e-08b69167d6c0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:28:42.734641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ed944-529d-4fc9-b04e-08b69167d6c0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:28:42.738370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9176138-6e3e-4f09-b482-429fc0903033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:28:42.739008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9176138-6e3e-4f09-b482-429fc0903033 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:28:42.739317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9176138-6e3e-4f09-b482-429fc0903033 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:28:42.739603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9176138-6e3e-4f09-b482-429fc0903033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:28:42.748564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9176138-6e3e-4f09-b482-429fc0903033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:28:42.749990 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9176138-6e3e-4f09-b482-429fc0903033 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:28:42.759349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 748/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:28:52.669354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=33 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:28:52.755544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=33 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:30:31.078907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d5a0d-1cbb-4886-b9f2-6aecefc2e0c3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.079888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d5a0d-1cbb-4886-b9f2-6aecefc2e0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.079888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d5a0d-1cbb-4886-b9f2-6aecefc2e0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.109650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-018d5a0d-1cbb-4886-b9f2-6aecefc2e0c3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:30:31.110870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-018d5a0d-1cbb-4886-b9f2-6aecefc2e0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:30:31.121333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 746/1494] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:30:31.130876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cbbf7-295a-4b58-993e-050dc596ade6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.131397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cbbf7-295a-4b58-993e-050dc596ade6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.131500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5cbbf7-295a-4b58-993e-050dc596ade6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.152064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1c5cbbf7-295a-4b58-993e-050dc596ade6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:30:31.153297 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1c5cbbf7-295a-4b58-993e-050dc596ade6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:30:31.162364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 749/1495] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:30:31.167143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d5a0d-1cbb-4886-b9f2-6aecefc2e0c3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:31.181807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d5a0d-1cbb-4886-b9f2-6aecefc2e0c3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:31.190973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:31.192074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.192533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.192983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.194154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:30:31.194656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:31.196338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:30:31.196338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jul 02 03:30:31.196338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: Jul 02 03:30:31.197909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:31.211238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 747/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:30:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:30:31.222089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cbbf7-295a-4b58-993e-050dc596ade6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:31.232379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5cbbf7-295a-4b58-993e-050dc596ade6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:31.238939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faca728-b84f-4cee-b6a1-0b4b429c8aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:31.239629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faca728-b84f-4cee-b6a1-0b4b429c8aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.239913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faca728-b84f-4cee-b6a1-0b4b429c8aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.240188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faca728-b84f-4cee-b6a1-0b4b429c8aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.240875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faca728-b84f-4cee-b6a1-0b4b429c8aa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:31.242692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-0faca728-b84f-4cee-b6a1-0b4b429c8aa7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:30:31.242692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jul 02 03:30:31.242692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: Jul 02 03:30:31.243924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faca728-b84f-4cee-b6a1-0b4b429c8aa7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:31.295634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 750/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:30:31 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:30:31.302418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:31.315532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a9c25-ee2d-4c4d-a14f-fc4a739a26ba admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:31.374294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8906c-3ce1-4d8d-b4ca-5834cd616e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:31.374924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8906c-3ce1-4d8d-b4ca-5834cd616e3f admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.375208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8906c-3ce1-4d8d-b4ca-5834cd616e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.375483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8906c-3ce1-4d8d-b4ca-5834cd616e3f admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.376175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8906c-3ce1-4d8d-b4ca-5834cd616e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:31.377460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-f6c8906c-3ce1-4d8d-b4ca-5834cd616e3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:30:31.377460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jul 02 03:30:31.377460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: Jul 02 03:30:31.378683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c8906c-3ce1-4d8d-b4ca-5834cd616e3f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:31.403275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 748/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:30:31 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:30:31.407473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faca728-b84f-4cee-b6a1-0b4b429c8aa7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:31.417991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faca728-b84f-4cee-b6a1-0b4b429c8aa7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:31.422780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c39f1-ceff-4497-8694-85bb4ce15c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:31.423465 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c39f1-ceff-4497-8694-85bb4ce15c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.424053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c39f1-ceff-4497-8694-85bb4ce15c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.424310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c39f1-ceff-4497-8694-85bb4ce15c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.425053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c39f1-ceff-4497-8694-85bb4ce15c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:31.426060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-0f5c39f1-ceff-4497-8694-85bb4ce15c0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:30:31.426060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jul 02 03:30:31.426060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: Jul 02 03:30:31.426964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c39f1-ceff-4497-8694-85bb4ce15c0a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:31.440826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 751/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:30:31.444995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8906c-3ce1-4d8d-b4ca-5834cd616e3f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:31.455655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8906c-3ce1-4d8d-b4ca-5834cd616e3f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:31.460764 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442e4a2-97b1-45c7-a002-98fc08f00bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:31.461420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442e4a2-97b1-45c7-a002-98fc08f00bce admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.461693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442e4a2-97b1-45c7-a002-98fc08f00bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.461968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442e4a2-97b1-45c7-a002-98fc08f00bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.462690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442e4a2-97b1-45c7-a002-98fc08f00bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:31.463738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-e442e4a2-97b1-45c7-a002-98fc08f00bce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:30:31.463738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jul 02 03:30:31.463738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: Jul 02 03:30:31.464611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442e4a2-97b1-45c7-a002-98fc08f00bce admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:31.479067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 749/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:30:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:30:31.485062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c39f1-ceff-4497-8694-85bb4ce15c0a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:31.499529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c39f1-ceff-4497-8694-85bb4ce15c0a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:31.505363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2f13b-33ea-4f33-a6ff-1b7f99be34db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:31.506240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2f13b-33ea-4f33-a6ff-1b7f99be34db admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.506517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2f13b-33ea-4f33-a6ff-1b7f99be34db admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.506829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2f13b-33ea-4f33-a6ff-1b7f99be34db admin admin] PATH_INFO: `/v3/projects/3dca050aeff34ae8870f58f0ff1a5d8b/users/47716d8ba252477192ace5fb6fe7e1b7/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.524071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2f13b-33ea-4f33-a6ff-1b7f99be34db admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dca050aeff34ae8870f58f0ff1a5d8b, user_id=47716d8ba252477192ace5fb6fe7e1b7, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:31.525123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-84d2f13b-33ea-4f33-a6ff-1b7f99be34db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:30:31.525123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jul 02 03:30:31.525123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: Jul 02 03:30:31.527064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2f13b-33ea-4f33-a6ff-1b7f99be34db admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:31.546298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 752/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:30:31 2022] PUT /identity/v3/projects/3dca050aeff34ae8870f58f0ff1a5d8b/users/47716d8ba252477192ace5fb6fe7e1b7/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:30:31.559722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3311195-a093-4995-b139-0ec81c187815 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.560006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3311195-a093-4995-b139-0ec81c187815 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.560278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3311195-a093-4995-b139-0ec81c187815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.588249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f3311195-a093-4995-b139-0ec81c187815 None None] MFA Rules not processed for user `47716d8ba252477192ace5fb6fe7e1b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:30:31.589535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f3311195-a093-4995-b139-0ec81c187815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:30:31.666402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 750/1502] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:30:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:30:31.680233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70462aee-ccad-4c92-b548-a358c4d2cd45 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.680520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70462aee-ccad-4c92-b548-a358c4d2cd45 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.680794 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70462aee-ccad-4c92-b548-a358c4d2cd45 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.682231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 753/1503] 10.222.0.42 () {54 vars in 963 bytes} [Sat Jul 2 03:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:30:31.687896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433d244-873d-49f7-b9c3-6ff693639000 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.688218 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433d244-873d-49f7-b9c3-6ff693639000 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.688497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433d244-873d-49f7-b9c3-6ff693639000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.706869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f433d244-873d-49f7-b9c3-6ff693639000 None None] MFA Rules not processed for user `e02b8a64165541889ca929b8038412bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:30:31.708109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f433d244-873d-49f7-b9c3-6ff693639000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:30:31.753916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:30:31.763322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 751/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Sat Jul 2 03:30:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:30:31.771316 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70462aee-ccad-4c92-b548-a358c4d2cd45 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:31.783174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70462aee-ccad-4c92-b548-a358c4d2cd45 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:31.787218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14db591-fa94-4eed-88da-41ad5f63c509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:31.787890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14db591-fa94-4eed-88da-41ad5f63c509 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.788165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14db591-fa94-4eed-88da-41ad5f63c509 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.788454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14db591-fa94-4eed-88da-41ad5f63c509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.797318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14db591-fa94-4eed-88da-41ad5f63c509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:31.799073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14db591-fa94-4eed-88da-41ad5f63c509 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:31.807878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 754/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:30:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:30:31.853336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461674c-2b94-4e27-a5d1-3fcd3ee5a221 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.853631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461674c-2b94-4e27-a5d1-3fcd3ee5a221 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.853907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461674c-2b94-4e27-a5d1-3fcd3ee5a221 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.855228 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 752/1506] 10.222.0.42 () {52 vars in 829 bytes} [Sat Jul 2 03:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:30:31.859186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14db591-fa94-4eed-88da-41ad5f63c509 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:31.924543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14db591-fa94-4eed-88da-41ad5f63c509 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:31.929114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:31.930003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:31.930283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:31.930556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] PATH_INFO: `/v3/users/47716d8ba252477192ace5fb6fe7e1b7` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:31.931938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47716d8ba252477192ace5fb6fe7e1b7)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:31.932981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:30:31.932981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jul 02 03:30:31.932981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: Jul 02 03:30:31.934450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:32.007725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47716d8ba252477192ace5fb6fe7e1b7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.008466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47716d8ba252477192ace5fb6fe7e1b7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.020550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47716d8ba252477192ace5fb6fe7e1b7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.021413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47716d8ba252477192ace5fb6fe7e1b7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.045138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47716d8ba252477192ace5fb6fe7e1b7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.058745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:30:32.070720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 755/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:30:31 2022] DELETE /identity/v3/users/47716d8ba252477192ace5fb6fe7e1b7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:30:32.075632 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cfcbd-c3a7-4a83-b9a3-4ce14242d060 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:32.075917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cfcbd-c3a7-4a83-b9a3-4ce14242d060 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:32.076198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cfcbd-c3a7-4a83-b9a3-4ce14242d060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:32.094744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a87cfcbd-c3a7-4a83-b9a3-4ce14242d060 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:30:32.095996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a87cfcbd-c3a7-4a83-b9a3-4ce14242d060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:30:32.158624 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 753/1508] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:30:32.168485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:32.170128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:30:32.233483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f26f3b-f7d4-4a28-8452-fc03f9877ab1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:32.237415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0ced16-342b-4fab-868f-f498031f1878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:32.238047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ced16-342b-4fab-868f-f498031f1878 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:32.238326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ced16-342b-4fab-868f-f498031f1878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:32.238595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0ced16-342b-4fab-868f-f498031f1878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:32.248441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ced16-342b-4fab-868f-f498031f1878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:32.249865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0ced16-342b-4fab-868f-f498031f1878 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:32.259260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 756/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:30:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:30:32.596332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87cfcbd-c3a7-4a83-b9a3-4ce14242d060 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:30:32.598691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a87cfcbd-c3a7-4a83-b9a3-4ce14242d060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:30:32.613620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87cfcbd-c3a7-4a83-b9a3-4ce14242d060 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:30:32.620115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:30:32.621232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:30:32.621656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:30:32.622154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] PATH_INFO: `/v3/projects/3dca050aeff34ae8870f58f0ff1a5d8b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:30:32.623599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dca050aeff34ae8870f58f0ff1a5d8b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:30:32.625408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:30:32.625408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jul 02 03:30:32.625408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: Jul 02 03:30:32.627119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:30:32.692067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dca050aeff34ae8870f58f0ff1a5d8b', 'request_id': 'req-9f277261-f680-49aa-b810-32f5d872bb6d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.699116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dca050aeff34ae8870f58f0ff1a5d8b', 'request_id': 'req-9f277261-f680-49aa-b810-32f5d872bb6d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.712732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dca050aeff34ae8870f58f0ff1a5d8b', 'request_id': 'req-9f277261-f680-49aa-b810-32f5d872bb6d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.728338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] The token cache is being invalidate because project 3dca050aeff34ae8870f58f0ff1a5d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:30:32.728653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.729530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9f277261-f680-49aa-b810-32f5d872bb6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:30:32.731365 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 754/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:30:32 2022] DELETE /identity/v3/projects/3dca050aeff34ae8870f58f0ff1a5d8b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:30:42.255076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=29 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:30:42.725395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:32:09.812428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae79bf-ec62-4905-97cd-1041db6a8d5f None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:09.813349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae79bf-ec62-4905-97cd-1041db6a8d5f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:09.813349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae79bf-ec62-4905-97cd-1041db6a8d5f None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:09.815647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:32:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:32:09.821356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca7b8-6e8d-4ba4-bb4e-17cf59ada34c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:09.821946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca7b8-6e8d-4ba4-bb4e-17cf59ada34c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:09.822263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ca7b8-6e8d-4ba4-bb4e-17cf59ada34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:09.845781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d75ca7b8-6e8d-4ba4-bb4e-17cf59ada34c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:09.847068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d75ca7b8-6e8d-4ba4-bb4e-17cf59ada34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:09.920376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:10.073872 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae79bf-ec62-4905-97cd-1041db6a8d5f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:32:10.077143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-86ae79bf-ec62-4905-97cd-1041db6a8d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:10.177195 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae79bf-ec62-4905-97cd-1041db6a8d5f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:32:10.184016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e79e4-520a-44e6-8dc5-05cbd472f519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:32:10.184689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e79e4-520a-44e6-8dc5-05cbd472f519 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:10.184981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e79e4-520a-44e6-8dc5-05cbd472f519 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:10.185257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e79e4-520a-44e6-8dc5-05cbd472f519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:10.195710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e79e4-520a-44e6-8dc5-05cbd472f519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:32:10.197353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e79e4-520a-44e6-8dc5-05cbd472f519 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:32:10.207848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 758/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:32:11.056322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac2f5f-4ca0-4649-b212-b6b6249c9020 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:11.056898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac2f5f-4ca0-4649-b212-b6b6249c9020 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:11.056898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac2f5f-4ca0-4649-b212-b6b6249c9020 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:11.058275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:32:11.064376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1adc1-548e-4fe2-bebb-829628901071 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:11.065050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1adc1-548e-4fe2-bebb-829628901071 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:11.065207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1adc1-548e-4fe2-bebb-829628901071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:11.092081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-88f1adc1-548e-4fe2-bebb-829628901071 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:11.093289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-88f1adc1-548e-4fe2-bebb-829628901071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:11.101805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:11.256370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac2f5f-4ca0-4649-b212-b6b6249c9020 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:32:11.279045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac2f5f-4ca0-4649-b212-b6b6249c9020 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:32:11.285652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03e856-9724-4f7b-b719-f32ad0a47ab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:32:11.286286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03e856-9724-4f7b-b719-f32ad0a47ab7 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:11.286566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03e856-9724-4f7b-b719-f32ad0a47ab7 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:11.286879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03e856-9724-4f7b-b719-f32ad0a47ab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:11.295576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03e856-9724-4f7b-b719-f32ad0a47ab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:32:11.297720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03e856-9724-4f7b-b719-f32ad0a47ab7 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:32:11.308038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 757/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:32:12.074432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530657b-f6e0-4ab4-bf23-b4beeaa4639b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:12.075029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530657b-f6e0-4ab4-bf23-b4beeaa4639b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:12.075029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530657b-f6e0-4ab4-bf23-b4beeaa4639b None None] PATH_INFO: `/v3` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:12.076352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 760/1517] 10.222.0.26 () {54 vars in 925 bytes} [Sat Jul 2 03:32:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:32:12.083899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d2442-6dae-4618-a82c-e033b11912da None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:12.084384 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d2442-6dae-4618-a82c-e033b11912da None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:12.084478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d2442-6dae-4618-a82c-e033b11912da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:12.104112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1c3d2442-6dae-4618-a82c-e033b11912da None None] MFA Rules not processed for user `6ee7acbd4c5545c1885faffb71f6306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:12.105352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1c3d2442-6dae-4618-a82c-e033b11912da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:12.175970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 758/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Jul 2 03:32:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:21.086255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=37 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:32:21.324780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dd487-eb84-4fbf-a3a4-b6f09d956447 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:21.324998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dd487-eb84-4fbf-a3a4-b6f09d956447 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:21.325258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dd487-eb84-4fbf-a3a4-b6f09d956447 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:21.327025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 761/1519] 10.222.0.42 () {54 vars in 960 bytes} [Sat Jul 2 03:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:32:21.335111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963b60c-7f93-45d6-a257-36caebfe6049 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:21.335978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963b60c-7f93-45d6-a257-36caebfe6049 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:21.336124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963b60c-7f93-45d6-a257-36caebfe6049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:21.356288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b963b60c-7f93-45d6-a257-36caebfe6049 None None] MFA Rules not processed for user `6ee7acbd4c5545c1885faffb71f6306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:21.357543 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b963b60c-7f93-45d6-a257-36caebfe6049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:21.367984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 759/1520] 10.222.0.42 () {58 vars in 1057 bytes} [Sat Jul 2 03:32:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:21.376075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dd487-eb84-4fbf-a3a4-b6f09d956447 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:32:21.393148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dd487-eb84-4fbf-a3a4-b6f09d956447 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:32:21.400577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53895060-38ea-4a4c-b1ce-8df870f93dbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:32:21.401356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53895060-38ea-4a4c-b1ce-8df870f93dbf service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:21.401751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53895060-38ea-4a4c-b1ce-8df870f93dbf service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:21.402192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53895060-38ea-4a4c-b1ce-8df870f93dbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:21.411611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53895060-38ea-4a4c-b1ce-8df870f93dbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:32:21.413218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53895060-38ea-4a4c-b1ce-8df870f93dbf service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:32:21.423535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 762/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:32:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:32:22.557484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c1cb8-7442-45cd-b310-82565496f7a1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:22.558091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c1cb8-7442-45cd-b310-82565496f7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:22.558091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c1cb8-7442-45cd-b310-82565496f7a1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:22.559492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:32:22.565400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f87270-0e5e-4648-b02a-c354a7890779 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:22.566131 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f87270-0e5e-4648-b02a-c354a7890779 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:22.566248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f87270-0e5e-4648-b02a-c354a7890779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:22.592426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-75f87270-0e5e-4648-b02a-c354a7890779 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:22.593612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-75f87270-0e5e-4648-b02a-c354a7890779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:22.602043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:22.768420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c1cb8-7442-45cd-b310-82565496f7a1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:32:22.786566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c1cb8-7442-45cd-b310-82565496f7a1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:32:22.793276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcede4-0b98-4d93-91dd-41102fd1b0a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:32:22.793846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcede4-0b98-4d93-91dd-41102fd1b0a1 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:22.794122 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcede4-0b98-4d93-91dd-41102fd1b0a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:22.794394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcede4-0b98-4d93-91dd-41102fd1b0a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:22.803094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfcede4-0b98-4d93-91dd-41102fd1b0a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:32:22.804720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfcede4-0b98-4d93-91dd-41102fd1b0a1 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:32:22.814212 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 761/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:32:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:32:23.633425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3957b-767e-471d-bd9f-3571045cd1f2 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:23.634027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3957b-767e-471d-bd9f-3571045cd1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:23.634027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3957b-767e-471d-bd9f-3571045cd1f2 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:23.635502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 2 03:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:32:23.644117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f559b41-07ff-4378-808f-3aa81bb216fa None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:23.644620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f559b41-07ff-4378-808f-3aa81bb216fa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:23.644713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f559b41-07ff-4378-808f-3aa81bb216fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:23.666494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5f559b41-07ff-4378-808f-3aa81bb216fa None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:23.667716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5f559b41-07ff-4378-808f-3aa81bb216fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:23.676884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 2 03:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:23.834604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3957b-767e-471d-bd9f-3571045cd1f2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:32:23.851743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3957b-767e-471d-bd9f-3571045cd1f2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:32:23.858327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f38d30-9ddb-41aa-9314-36fc62a38304 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:32:23.858966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38d30-9ddb-41aa-9314-36fc62a38304 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:23.859237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38d30-9ddb-41aa-9314-36fc62a38304 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:23.859540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f38d30-9ddb-41aa-9314-36fc62a38304 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:23.867725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f38d30-9ddb-41aa-9314-36fc62a38304 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:32:23.869314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f38d30-9ddb-41aa-9314-36fc62a38304 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:32:23.878773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 765/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:32:27.826043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f3ddd-2a5d-4639-ae05-733c81ecd4a7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:27.827009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f3ddd-2a5d-4639-ae05-733c81ecd4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:27.827009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f3ddd-2a5d-4639-ae05-733c81ecd4a7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:27.829630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:32:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:32:27.839318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608dea3-df86-4cac-bb24-aa1343d3ec69 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:27.839785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608dea3-df86-4cac-bb24-aa1343d3ec69 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:27.839890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608dea3-df86-4cac-bb24-aa1343d3ec69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:27.858930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4608dea3-df86-4cac-bb24-aa1343d3ec69 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:27.860200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4608dea3-df86-4cac-bb24-aa1343d3ec69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:27.869238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:27.910308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f3ddd-2a5d-4639-ae05-733c81ecd4a7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:32:27.913281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fc5f3ddd-2a5d-4639-ae05-733c81ecd4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:27.982560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f3ddd-2a5d-4639-ae05-733c81ecd4a7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:32:27.986934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262a94-2122-42e8-9428-229ebc8509bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:32:27.987609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262a94-2122-42e8-9428-229ebc8509bb service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:27.987868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262a94-2122-42e8-9428-229ebc8509bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:27.988138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262a94-2122-42e8-9428-229ebc8509bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:27.997769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d262a94-2122-42e8-9428-229ebc8509bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:32:27.999205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d262a94-2122-42e8-9428-229ebc8509bb service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:32:28.004165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:32:28.009203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 764/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:32:30.074019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bd161-693a-4160-ad61-791677be841c None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:30.074621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bd161-693a-4160-ad61-791677be841c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:30.074621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bd161-693a-4160-ad61-791677be841c None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:30.076047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:32:30.081403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2dede-34c9-4584-91cd-be3ac587cc78 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:30.081840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2dede-34c9-4584-91cd-be3ac587cc78 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:30.081952 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2dede-34c9-4584-91cd-be3ac587cc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:30.107191 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f7b2dede-34c9-4584-91cd-be3ac587cc78 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:30.108418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f7b2dede-34c9-4584-91cd-be3ac587cc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:30.121972 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:30.157690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bd161-693a-4160-ad61-791677be841c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:32:30.159512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a49bd161-693a-4160-ad61-791677be841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:30.174388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bd161-693a-4160-ad61-791677be841c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:32:30.178988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff9f8d-dbca-4d65-9a9c-ff0e894d38a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:32:30.179665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff9f8d-dbca-4d65-9a9c-ff0e894d38a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:30.179945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff9f8d-dbca-4d65-9a9c-ff0e894d38a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:30.180215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff9f8d-dbca-4d65-9a9c-ff0e894d38a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:30.189904 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff9f8d-dbca-4d65-9a9c-ff0e894d38a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:32:30.191347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff9f8d-dbca-4d65-9a9c-ff0e894d38a0 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:32:30.200975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 768/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:32:33.670460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04093f3e-07bb-4e83-b480-05733cf0a343 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:33.671061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04093f3e-07bb-4e83-b480-05733cf0a343 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:33.671061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04093f3e-07bb-4e83-b480-05733cf0a343 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:33.672763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:32:33.681590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b6fc4-d175-408d-be79-7fcbf9ff89e6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:33.682341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b6fc4-d175-408d-be79-7fcbf9ff89e6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:33.682501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b6fc4-d175-408d-be79-7fcbf9ff89e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:33.708541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1e1b6fc4-d175-408d-be79-7fcbf9ff89e6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:33.709754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1e1b6fc4-d175-408d-be79-7fcbf9ff89e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:33.719453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:32:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:34.026263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04093f3e-07bb-4e83-b480-05733cf0a343 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:32:34.028617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-04093f3e-07bb-4e83-b480-05733cf0a343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:34.090482 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04093f3e-07bb-4e83-b480-05733cf0a343 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:32:34.094559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65c908-eece-46e0-86c3-6a5b98b2ef65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:32:34.095200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65c908-eece-46e0-86c3-6a5b98b2ef65 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:34.095516 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65c908-eece-46e0-86c3-6a5b98b2ef65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:34.095790 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65c908-eece-46e0-86c3-6a5b98b2ef65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:34.105464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65c908-eece-46e0-86c3-6a5b98b2ef65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:32:34.106943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65c908-eece-46e0-86c3-6a5b98b2ef65 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:32:34.117156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 767/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:32:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:32:36.160367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faa0c5-9be2-4506-9b9f-b38ff08e11fd None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:36.160917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faa0c5-9be2-4506-9b9f-b38ff08e11fd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:36.160917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faa0c5-9be2-4506-9b9f-b38ff08e11fd None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:36.162312 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 2 03:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:32:36.168866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf165c-98e1-4ed8-9d5e-1acb02eb904d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:36.169308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf165c-98e1-4ed8-9d5e-1acb02eb904d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:36.169457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf165c-98e1-4ed8-9d5e-1acb02eb904d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:36.198818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-aaaf165c-98e1-4ed8-9d5e-1acb02eb904d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:32:36.200729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aaaf165c-98e1-4ed8-9d5e-1acb02eb904d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:36.215126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 2 03:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:32:36.517255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faa0c5-9be2-4506-9b9f-b38ff08e11fd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:32:36.520894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-70faa0c5-9be2-4506-9b9f-b38ff08e11fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:32:36.539077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faa0c5-9be2-4506-9b9f-b38ff08e11fd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:32:36.543599 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca14a11-dcfa-4dbc-92c1-bac21c67685b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:32:36.544240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca14a11-dcfa-4dbc-92c1-bac21c67685b service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:32:36.544525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca14a11-dcfa-4dbc-92c1-bac21c67685b service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:32:36.544821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca14a11-dcfa-4dbc-92c1-bac21c67685b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:32:36.554855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca14a11-dcfa-4dbc-92c1-bac21c67685b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:32:36.556141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca14a11-dcfa-4dbc-92c1-bac21c67685b service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:32:36.566832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 771/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:32:46.189691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=32 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:32:46.562275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=33 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:05.801855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d9ce0-812e-4944-b29f-7488590aac6d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:05.802550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d9ce0-812e-4944-b29f-7488590aac6d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:05.802550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d9ce0-812e-4944-b29f-7488590aac6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:05.824546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ce0d9ce0-812e-4944-b29f-7488590aac6d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:05.825873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ce0d9ce0-812e-4944-b29f-7488590aac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:05.836153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 769/1540] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:05.845056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529c2c5-ddc6-47c2-8777-f11657f3445d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:05.845632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529c2c5-ddc6-47c2-8777-f11657f3445d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:05.845632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529c2c5-ddc6-47c2-8777-f11657f3445d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:05.878872 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6529c2c5-ddc6-47c2-8777-f11657f3445d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:05.880409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6529c2c5-ddc6-47c2-8777-f11657f3445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:05.892918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 772/1541] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:05.897708 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d9ce0-812e-4944-b29f-7488590aac6d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:05.908349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d9ce0-812e-4944-b29f-7488590aac6d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:05.919465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46febc0-0e4a-4e04-bf08-18352e61db76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:05.920153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46febc0-0e4a-4e04-bf08-18352e61db76 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:05.921848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46febc0-0e4a-4e04-bf08-18352e61db76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:05.922136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46febc0-0e4a-4e04-bf08-18352e61db76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:05.922875 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46febc0-0e4a-4e04-bf08-18352e61db76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:05.923183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46febc0-0e4a-4e04-bf08-18352e61db76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:05.924848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46febc0-0e4a-4e04-bf08-18352e61db76 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:05.942699 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 770/1542] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:05.951494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529c2c5-ddc6-47c2-8777-f11657f3445d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:05.962578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146ee64-2828-4798-9696-1ad3dfa9d3ea None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:05.962888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146ee64-2828-4798-9696-1ad3dfa9d3ea None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:05.963686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146ee64-2828-4798-9696-1ad3dfa9d3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:05.969831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529c2c5-ddc6-47c2-8777-f11657f3445d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:05.982074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78b1c3-c469-4ea1-822a-7f0425a0ceb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:05.982759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78b1c3-c469-4ea1-822a-7f0425a0ceb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:05.983041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78b1c3-c469-4ea1-822a-7f0425a0ceb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:05.983333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78b1c3-c469-4ea1-822a-7f0425a0ceb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:05.984041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78b1c3-c469-4ea1-822a-7f0425a0ceb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:05.985981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78b1c3-c469-4ea1-822a-7f0425a0ceb3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:05.990835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a146ee64-2828-4798-9696-1ad3dfa9d3ea None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:05.992070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a146ee64-2828-4798-9696-1ad3dfa9d3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.005863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 771/1543] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:06.015182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1581ef-8345-4974-8481-8489a2194e14 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.015518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1581ef-8345-4974-8481-8489a2194e14 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.015770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1581ef-8345-4974-8481-8489a2194e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.041467 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 773/1544] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:05 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:06.043085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fa1581ef-8345-4974-8481-8489a2194e14 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.044586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fa1581ef-8345-4974-8481-8489a2194e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.045441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78b1c3-c469-4ea1-822a-7f0425a0ceb3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.123902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 772/1545] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.125270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5ad04-c679-462e-a09d-84df3caa2a56 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.125562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5ad04-c679-462e-a09d-84df3caa2a56 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.126008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5ad04-c679-462e-a09d-84df3caa2a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.154721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78b1c3-c469-4ea1-822a-7f0425a0ceb3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.157193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7af5ad04-c679-462e-a09d-84df3caa2a56 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.158577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7af5ad04-c679-462e-a09d-84df3caa2a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.159910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd70494-f463-4880-bb82-82df66558f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.160721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70494-f463-4880-bb82-82df66558f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.161010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70494-f463-4880-bb82-82df66558f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.161290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70494-f463-4880-bb82-82df66558f33 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.161991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd70494-f463-4880-bb82-82df66558f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.163148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-9fd70494-f463-4880-bb82-82df66558f33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:34:06.163148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jul 02 03:34:06.163148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: Jul 02 03:34:06.164493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd70494-f463-4880-bb82-82df66558f33 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.178962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 773/1546] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.180477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcec66-6152-420e-9e2c-7d2c210f4bbd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.181328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcec66-6152-420e-9e2c-7d2c210f4bbd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.181328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcec66-6152-420e-9e2c-7d2c210f4bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.195926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 774/1547] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:06.197112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9ca2e-c60a-467f-a57a-731433c8892c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.199918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9ca2e-c60a-467f-a57a-731433c8892c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.199918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9ca2e-c60a-467f-a57a-731433c8892c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.206776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f4bcec66-6152-420e-9e2c-7d2c210f4bbd None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.208154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f4bcec66-6152-420e-9e2c-7d2c210f4bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.219324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 774/1548] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.221589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25d5e3-c01f-48a3-9e06-8e9a8b029f40 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.221710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1fc9ca2e-c60a-467f-a57a-731433c8892c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.221811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25d5e3-c01f-48a3-9e06-8e9a8b029f40 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.221811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25d5e3-c01f-48a3-9e06-8e9a8b029f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.222222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1fc9ca2e-c60a-467f-a57a-731433c8892c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.230924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 775/1549] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.231809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9ca2e-c60a-467f-a57a-731433c8892c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.240530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2e25d5e3-c01f-48a3-9e06-8e9a8b029f40 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.241759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2e25d5e3-c01f-48a3-9e06-8e9a8b029f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.241914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9ca2e-c60a-467f-a57a-731433c8892c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.246483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2dd6a-629e-4fe1-b81c-bdef3e917c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.247172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2dd6a-629e-4fe1-b81c-bdef3e917c24 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.247494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2dd6a-629e-4fe1-b81c-bdef3e917c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.247771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2dd6a-629e-4fe1-b81c-bdef3e917c24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.248546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2dd6a-629e-4fe1-b81c-bdef3e917c24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:06.248856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2dd6a-629e-4fe1-b81c-bdef3e917c24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.249877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-d1d2dd6a-629e-4fe1-b81c-bdef3e917c24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:34:06.249877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jul 02 03:34:06.249877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: Jul 02 03:34:06.250545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 775/1550] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.253555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2dd6a-629e-4fe1-b81c-bdef3e917c24 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.253710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959589fd-4acb-4e43-aad8-13246edc5d1e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.253710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959589fd-4acb-4e43-aad8-13246edc5d1e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.253710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959589fd-4acb-4e43-aad8-13246edc5d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.264799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 776/1551] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:06.266071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bdbba-134b-4e0d-9685-c77f61eb031d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.266381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bdbba-134b-4e0d-9685-c77f61eb031d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.266692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bdbba-134b-4e0d-9685-c77f61eb031d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.274034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-959589fd-4acb-4e43-aad8-13246edc5d1e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.275475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-959589fd-4acb-4e43-aad8-13246edc5d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.284137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 776/1552] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.284980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959589fd-4acb-4e43-aad8-13246edc5d1e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.290969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eb2bdbba-134b-4e0d-9685-c77f61eb031d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.292576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eb2bdbba-134b-4e0d-9685-c77f61eb031d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.295323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959589fd-4acb-4e43-aad8-13246edc5d1e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.300770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a8263-ebd6-41c0-94ad-23a812ba4f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.301315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 777/1553] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.305344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a8263-ebd6-41c0-94ad-23a812ba4f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.305344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a8263-ebd6-41c0-94ad-23a812ba4f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.305344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a8263-ebd6-41c0-94ad-23a812ba4f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.305674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae759f7f-3152-434d-b2c7-8a198d97a9e4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.305674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae759f7f-3152-434d-b2c7-8a198d97a9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.305859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a8263-ebd6-41c0-94ad-23a812ba4f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.305953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae759f7f-3152-434d-b2c7-8a198d97a9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.306081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a8263-ebd6-41c0-94ad-23a812ba4f99 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.316998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 777/1554] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:06.318275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895c2e3-b0ec-4907-aeaa-048f4e5c29a9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.318561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895c2e3-b0ec-4907-aeaa-048f4e5c29a9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.318854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895c2e3-b0ec-4907-aeaa-048f4e5c29a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.323180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ae759f7f-3152-434d-b2c7-8a198d97a9e4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.324507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ae759f7f-3152-434d-b2c7-8a198d97a9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.332740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 778/1555] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.333985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aced71a-f0bf-4a27-abf0-f06a12268c45 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.334278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aced71a-f0bf-4a27-abf0-f06a12268c45 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.334551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aced71a-f0bf-4a27-abf0-f06a12268c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.339034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2895c2e3-b0ec-4907-aeaa-048f4e5c29a9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.340466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2895c2e3-b0ec-4907-aeaa-048f4e5c29a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.348915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 778/1556] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.351021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019da39a-2b0b-4980-82d3-9598f99000c2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.351330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019da39a-2b0b-4980-82d3-9598f99000c2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.351612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019da39a-2b0b-4980-82d3-9598f99000c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.353157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6aced71a-f0bf-4a27-abf0-f06a12268c45 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.354406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6aced71a-f0bf-4a27-abf0-f06a12268c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.362482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 779/1557] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.363186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aced71a-f0bf-4a27-abf0-f06a12268c45 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.372776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aced71a-f0bf-4a27-abf0-f06a12268c45 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.373458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-019da39a-2b0b-4980-82d3-9598f99000c2 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.374658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-019da39a-2b0b-4980-82d3-9598f99000c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.377502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cced5e-eef4-4245-813f-ad3fa1f107bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.378134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cced5e-eef4-4245-813f-ad3fa1f107bc admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.378399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cced5e-eef4-4245-813f-ad3fa1f107bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.378672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cced5e-eef4-4245-813f-ad3fa1f107bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.379391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cced5e-eef4-4245-813f-ad3fa1f107bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.381313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cced5e-eef4-4245-813f-ad3fa1f107bc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.382822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 779/1558] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.384121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ce0c6-eade-4f49-af96-5e7919303ed6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.384400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ce0c6-eade-4f49-af96-5e7919303ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.384683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ce0c6-eade-4f49-af96-5e7919303ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.402283 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d18ce0c6-eade-4f49-af96-5e7919303ed6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.403556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d18ce0c6-eade-4f49-af96-5e7919303ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.409299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 780/1559] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:06.410209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cced5e-eef4-4245-813f-ad3fa1f107bc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.415841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 780/1560] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.415841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ce0c6-eade-4f49-af96-5e7919303ed6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.423223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ce0c6-eade-4f49-af96-5e7919303ed6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.476992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cced5e-eef4-4245-813f-ad3fa1f107bc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.477923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c83ac-1d37-402e-96ef-d55ded9637a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.478727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c83ac-1d37-402e-96ef-d55ded9637a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.479021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c83ac-1d37-402e-96ef-d55ded9637a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.479326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c83ac-1d37-402e-96ef-d55ded9637a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.480078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c83ac-1d37-402e-96ef-d55ded9637a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.481686 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7798e-5178-4749-b6c5-892174354821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.481820 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c83ac-1d37-402e-96ef-d55ded9637a8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.482331 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7798e-5178-4749-b6c5-892174354821 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.482624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7798e-5178-4749-b6c5-892174354821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.482980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7798e-5178-4749-b6c5-892174354821 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.483641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7798e-5178-4749-b6c5-892174354821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:06.483939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7798e-5178-4749-b6c5-892174354821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.485548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7798e-5178-4749-b6c5-892174354821 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.487184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:06.492359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 781/1561] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:06.493798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede6f8-466c-433b-aca9-429c9b1beb9d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.494072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede6f8-466c-433b-aca9-429c9b1beb9d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.494460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ede6f8-466c-433b-aca9-429c9b1beb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.502027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 781/1562] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:06.503571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7798e-5178-4749-b6c5-892174354821 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.514101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7798e-5178-4749-b6c5-892174354821 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.518766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa5a2b-c79b-4734-b93e-d50027fa6359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.519477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5a2b-c79b-4734-b93e-d50027fa6359 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.519757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5a2b-c79b-4734-b93e-d50027fa6359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.520031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5a2b-c79b-4734-b93e-d50027fa6359 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.520757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa5a2b-c79b-4734-b93e-d50027fa6359 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:06.521061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa5a2b-c79b-4734-b93e-d50027fa6359 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.521442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-15ede6f8-466c-433b-aca9-429c9b1beb9d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.522631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-15ede6f8-466c-433b-aca9-429c9b1beb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.522760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa5a2b-c79b-4734-b93e-d50027fa6359 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.530710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 782/1563] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:06.534602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede6f8-466c-433b-aca9-429c9b1beb9d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.535543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 782/1564] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:06.536254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa5a2b-c79b-4734-b93e-d50027fa6359 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.543983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ede6f8-466c-433b-aca9-429c9b1beb9d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.547639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa5a2b-c79b-4734-b93e-d50027fa6359 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.549252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b3601-a126-4090-b95a-fcb0748eabce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.549920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b3601-a126-4090-b95a-fcb0748eabce admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.550313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b3601-a126-4090-b95a-fcb0748eabce admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.550590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b3601-a126-4090-b95a-fcb0748eabce admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.551334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b3601-a126-4090-b95a-fcb0748eabce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:06.551645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b3601-a126-4090-b95a-fcb0748eabce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.553255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b3601-a126-4090-b95a-fcb0748eabce admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.553361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f37e95-c446-478a-8007-1f0ee77607ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.553962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f37e95-c446-478a-8007-1f0ee77607ac admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.554242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f37e95-c446-478a-8007-1f0ee77607ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.554516 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f37e95-c446-478a-8007-1f0ee77607ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.554935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:06.555240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f37e95-c446-478a-8007-1f0ee77607ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:06.555598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f37e95-c446-478a-8007-1f0ee77607ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.557303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f37e95-c446-478a-8007-1f0ee77607ac admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.572350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 783/1565] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:06.572350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b3601-a126-4090-b95a-fcb0748eabce admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.576363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 783/1566] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:06.577247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f37e95-c446-478a-8007-1f0ee77607ac admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.577645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b3601-a126-4090-b95a-fcb0748eabce admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.582362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bac32-36f5-433b-99cd-34acf5321f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.583609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bac32-36f5-433b-99cd-34acf5321f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.583892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bac32-36f5-433b-99cd-34acf5321f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.584167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bac32-36f5-433b-99cd-34acf5321f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.584852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bac32-36f5-433b-99cd-34acf5321f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.586944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bac32-36f5-433b-99cd-34acf5321f58 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.588198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f37e95-c446-478a-8007-1f0ee77607ac admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.592847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68978c35-9b0b-44ca-a873-67327f733139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.593519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68978c35-9b0b-44ca-a873-67327f733139 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.593800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68978c35-9b0b-44ca-a873-67327f733139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.594072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68978c35-9b0b-44ca-a873-67327f733139 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.594784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68978c35-9b0b-44ca-a873-67327f733139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:06.595177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68978c35-9b0b-44ca-a873-67327f733139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.596879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68978c35-9b0b-44ca-a873-67327f733139 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.608822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 784/1567] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:06.609619 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bac32-36f5-433b-99cd-34acf5321f58 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.617819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 784/1568] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:06.617819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68978c35-9b0b-44ca-a873-67327f733139 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.619362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bac32-36f5-433b-99cd-34acf5321f58 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.622436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68978c35-9b0b-44ca-a873-67327f733139 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.624458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d87f1b-74c4-4195-8ba8-0e989a0f63bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.625487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d87f1b-74c4-4195-8ba8-0e989a0f63bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.625937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d87f1b-74c4-4195-8ba8-0e989a0f63bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.626216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d87f1b-74c4-4195-8ba8-0e989a0f63bb admin admin] PATH_INFO: `/v3/projects/dedc7a5a95d14f60b1a73198c8b5309b/users/9028859afbab40df9fc4e5ea6d876931/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.626944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76227d7-9ff4-4f1c-882c-87fd9b30e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.627613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76227d7-9ff4-4f1c-882c-87fd9b30e69f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.627895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76227d7-9ff4-4f1c-882c-87fd9b30e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.628170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76227d7-9ff4-4f1c-882c-87fd9b30e69f admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.628850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76227d7-9ff4-4f1c-882c-87fd9b30e69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.630739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76227d7-9ff4-4f1c-882c-87fd9b30e69f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.641594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d87f1b-74c4-4195-8ba8-0e989a0f63bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dedc7a5a95d14f60b1a73198c8b5309b, user_id=9028859afbab40df9fc4e5ea6d876931, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.642649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-00d87f1b-74c4-4195-8ba8-0e989a0f63bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:34:06.642649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jul 02 03:34:06.642649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: Jul 02 03:34:06.644620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d87f1b-74c4-4195-8ba8-0e989a0f63bb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.660720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 785/1569] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:06.660949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 785/1570] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:06 2022] PUT /identity/v3/projects/dedc7a5a95d14f60b1a73198c8b5309b/users/9028859afbab40df9fc4e5ea6d876931/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:06.661052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76227d7-9ff4-4f1c-882c-87fd9b30e69f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.661156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d87f1b-74c4-4195-8ba8-0e989a0f63bb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.721820 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d87f1b-74c4-4195-8ba8-0e989a0f63bb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.722099 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76227d7-9ff4-4f1c-882c-87fd9b30e69f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.726630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9abb7d-0ca6-4d18-8185-6cccdb4d2be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.726771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c8078-ed1f-4aff-9fe5-13b0ab5e7731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.727399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9abb7d-0ca6-4d18-8185-6cccdb4d2be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.727525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c8078-ed1f-4aff-9fe5-13b0ab5e7731 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.727724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9abb7d-0ca6-4d18-8185-6cccdb4d2be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.727838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c8078-ed1f-4aff-9fe5-13b0ab5e7731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.728016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c8078-ed1f-4aff-9fe5-13b0ab5e7731 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.728121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9abb7d-0ca6-4d18-8185-6cccdb4d2be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.728765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c8078-ed1f-4aff-9fe5-13b0ab5e7731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:06.728864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9abb7d-0ca6-4d18-8185-6cccdb4d2be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.729067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c8078-ed1f-4aff-9fe5-13b0ab5e7731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.729867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-be9abb7d-0ca6-4d18-8185-6cccdb4d2be1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:34:06.729867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jul 02 03:34:06.729867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: Jul 02 03:34:06.730680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c8078-ed1f-4aff-9fe5-13b0ab5e7731 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.731060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9abb7d-0ca6-4d18-8185-6cccdb4d2be1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.743190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 786/1571] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:06.744219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c8078-ed1f-4aff-9fe5-13b0ab5e7731 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.754321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c8078-ed1f-4aff-9fe5-13b0ab5e7731 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.759063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d4c2e-a9d5-4bcc-bc8a-33f030f63b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.759749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4c2e-a9d5-4bcc-bc8a-33f030f63b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.760036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4c2e-a9d5-4bcc-bc8a-33f030f63b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.760310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4c2e-a9d5-4bcc-bc8a-33f030f63b26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.760997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d4c2e-a9d5-4bcc-bc8a-33f030f63b26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.766032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 786/1572] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:06.766188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d4c2e-a9d5-4bcc-bc8a-33f030f63b26 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.766285 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9abb7d-0ca6-4d18-8185-6cccdb4d2be1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.793581 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 787/1573] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:06.794935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d4c2e-a9d5-4bcc-bc8a-33f030f63b26 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.831584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9abb7d-0ca6-4d18-8185-6cccdb4d2be1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.856298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d4c2e-a9d5-4bcc-bc8a-33f030f63b26 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.861035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd9124-6568-4c3d-bb4b-0f42de6ea782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.861769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd9124-6568-4c3d-bb4b-0f42de6ea782 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.862042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd9124-6568-4c3d-bb4b-0f42de6ea782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.862329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd9124-6568-4c3d-bb4b-0f42de6ea782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.863085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd9124-6568-4c3d-bb4b-0f42de6ea782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.864753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd9124-6568-4c3d-bb4b-0f42de6ea782 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.870249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:06.873898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:06.877261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 788/1574] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:06.878773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd9124-6568-4c3d-bb4b-0f42de6ea782 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.889118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd9124-6568-4c3d-bb4b-0f42de6ea782 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:06.891710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc14a5-0f14-4629-ba68-49d04ec1fd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.892385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc14a5-0f14-4629-ba68-49d04ec1fd5e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.892736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc14a5-0f14-4629-ba68-49d04ec1fd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.892943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc14a5-0f14-4629-ba68-49d04ec1fd5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.893635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc14a5-0f14-4629-ba68-49d04ec1fd5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.894456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b951be5-1ab1-4768-a1cc-b19500183d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:06.895097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b951be5-1ab1-4768-a1cc-b19500183d42 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.895407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b951be5-1ab1-4768-a1cc-b19500183d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.895587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc14a5-0f14-4629-ba68-49d04ec1fd5e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.895695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b951be5-1ab1-4768-a1cc-b19500183d42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.896344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b951be5-1ab1-4768-a1cc-b19500183d42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:06.898271 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b951be5-1ab1-4768-a1cc-b19500183d42 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:06.922779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 787/1575] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/projects => generated 389 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:06.923621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc14a5-0f14-4629-ba68-49d04ec1fd5e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:06.926412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 789/1576] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:06.927898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dbb31-ce1a-4f05-b745-d868de285f9a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:06.928289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dbb31-ce1a-4f05-b745-d868de285f9a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:06.928858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8dbb31-ce1a-4f05-b745-d868de285f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:06.946763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3a8dbb31-ce1a-4f05-b745-d868de285f9a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:06.948034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a8dbb31-ce1a-4f05-b745-d868de285f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:06.988786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc14a5-0f14-4629-ba68-49d04ec1fd5e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.004985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 790/1577] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:07.005809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dbb31-ce1a-4f05-b745-d868de285f9a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.015277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8dbb31-ce1a-4f05-b745-d868de285f9a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.019704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f0bd-042a-4bbb-9245-f73bf7413c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.020358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f0bd-042a-4bbb-9245-f73bf7413c5a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.020640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f0bd-042a-4bbb-9245-f73bf7413c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.020915 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137f0bd-042a-4bbb-9245-f73bf7413c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.021605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f0bd-042a-4bbb-9245-f73bf7413c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.023560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137f0bd-042a-4bbb-9245-f73bf7413c5a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.047683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604d24f-7798-42ec-9177-bd12b61beeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.048452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604d24f-7798-42ec-9177-bd12b61beeb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.048808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604d24f-7798-42ec-9177-bd12b61beeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.049084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604d24f-7798-42ec-9177-bd12b61beeb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.049770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604d24f-7798-42ec-9177-bd12b61beeb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.055020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 791/1578] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:07 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:07.055195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604d24f-7798-42ec-9177-bd12b61beeb7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.055404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f0bd-042a-4bbb-9245-f73bf7413c5a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.077936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 788/1579] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:06 2022] POST /identity/v3/users => generated 279 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:07.077936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604d24f-7798-42ec-9177-bd12b61beeb7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.084838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604d24f-7798-42ec-9177-bd12b61beeb7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.117662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137f0bd-042a-4bbb-9245-f73bf7413c5a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.122722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfd2df-dfe1-43d9-be44-e9c6bd9a3125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.123373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfd2df-dfe1-43d9-be44-e9c6bd9a3125 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.123645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfd2df-dfe1-43d9-be44-e9c6bd9a3125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.123923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfd2df-dfe1-43d9-be44-e9c6bd9a3125 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.124617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfd2df-dfe1-43d9-be44-e9c6bd9a3125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.126596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfd2df-dfe1-43d9-be44-e9c6bd9a3125 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.148496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdcc64-f59c-48ea-9d6f-bc67e904c612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.149254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdcc64-f59c-48ea-9d6f-bc67e904c612 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.149536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdcc64-f59c-48ea-9d6f-bc67e904c612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.149731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 792/1580] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:07 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:07.150278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdcc64-f59c-48ea-9d6f-bc67e904c612 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.150908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdcc64-f59c-48ea-9d6f-bc67e904c612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.151031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfd2df-dfe1-43d9-be44-e9c6bd9a3125 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.152984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdcc64-f59c-48ea-9d6f-bc67e904c612 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.160447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfd2df-dfe1-43d9-be44-e9c6bd9a3125 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.163140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:07.166521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5507aa-001c-4667-b14c-797e79f3c555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.167260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5507aa-001c-4667-b14c-797e79f3c555 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.167635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5507aa-001c-4667-b14c-797e79f3c555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.167913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5507aa-001c-4667-b14c-797e79f3c555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.168996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5507aa-001c-4667-b14c-797e79f3c555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.171018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5507aa-001c-4667-b14c-797e79f3c555 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.176876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 789/1581] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:07 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:07.177549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdcc64-f59c-48ea-9d6f-bc67e904c612 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.184877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 793/1582] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.185954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5507aa-001c-4667-b14c-797e79f3c555 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.188046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdcc64-f59c-48ea-9d6f-bc67e904c612 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.192563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3ed83-00c6-4b98-a160-43ea00c977ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.193229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3ed83-00c6-4b98-a160-43ea00c977ff admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.193511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3ed83-00c6-4b98-a160-43ea00c977ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.193787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3ed83-00c6-4b98-a160-43ea00c977ff admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.194469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3ed83-00c6-4b98-a160-43ea00c977ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.196150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5507aa-001c-4667-b14c-797e79f3c555 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.196710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3ed83-00c6-4b98-a160-43ea00c977ff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.200651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d1261-1fcf-4e3f-bf03-45e2fd225f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.201302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d1261-1fcf-4e3f-bf03-45e2fd225f76 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.201583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d1261-1fcf-4e3f-bf03-45e2fd225f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.201856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d1261-1fcf-4e3f-bf03-45e2fd225f76 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.202559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d1261-1fcf-4e3f-bf03-45e2fd225f76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.204515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d1261-1fcf-4e3f-bf03-45e2fd225f76 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.217066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 790/1583] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:07 2022] POST /identity/v3/users => generated 295 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:07.217848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3ed83-00c6-4b98-a160-43ea00c977ff admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.231989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 794/1584] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:07 2022] POST /identity/v3/users => generated 301 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:07.231989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d1261-1fcf-4e3f-bf03-45e2fd225f76 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.239225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d1261-1fcf-4e3f-bf03-45e2fd225f76 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.244171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b989f-7daf-4c8b-ac85-a6f0ae46aac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.244860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b989f-7daf-4c8b-ac85-a6f0ae46aac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.245143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b989f-7daf-4c8b-ac85-a6f0ae46aac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.245416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b989f-7daf-4c8b-ac85-a6f0ae46aac2 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.246101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b989f-7daf-4c8b-ac85-a6f0ae46aac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.248132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b989f-7daf-4c8b-ac85-a6f0ae46aac2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.266257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:07.272321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 795/1585] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:07 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:07.273892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b989f-7daf-4c8b-ac85-a6f0ae46aac2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.277486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3ed83-00c6-4b98-a160-43ea00c977ff admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.282827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd615bba-1647-4880-b89a-09da686d3662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.283518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd615bba-1647-4880-b89a-09da686d3662 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.283799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd615bba-1647-4880-b89a-09da686d3662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.284073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd615bba-1647-4880-b89a-09da686d3662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.284684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b989f-7daf-4c8b-ac85-a6f0ae46aac2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.289764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bafbbc-5811-4fdb-a53e-34dea5b39671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.290592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bafbbc-5811-4fdb-a53e-34dea5b39671 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.290890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bafbbc-5811-4fdb-a53e-34dea5b39671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.291193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bafbbc-5811-4fdb-a53e-34dea5b39671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.292237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bafbbc-5811-4fdb-a53e-34dea5b39671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.294110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bafbbc-5811-4fdb-a53e-34dea5b39671 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.296009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd615bba-1647-4880-b89a-09da686d3662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.297707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd615bba-1647-4880-b89a-09da686d3662 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.304755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 796/1586] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.305675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bafbbc-5811-4fdb-a53e-34dea5b39671 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.311166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 791/1587] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:07.312196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd615bba-1647-4880-b89a-09da686d3662 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.315820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bafbbc-5811-4fdb-a53e-34dea5b39671 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.320735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edea741-54f1-4843-9912-d7e3a1d1d1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.321587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edea741-54f1-4843-9912-d7e3a1d1d1e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.321940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edea741-54f1-4843-9912-d7e3a1d1d1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.322053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd615bba-1647-4880-b89a-09da686d3662 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.322340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edea741-54f1-4843-9912-d7e3a1d1d1e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.323133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edea741-54f1-4843-9912-d7e3a1d1d1e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.325325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edea741-54f1-4843-9912-d7e3a1d1d1e6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.327046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfeed3-f80c-4e04-9d15-786de3d08e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.327946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfeed3-f80c-4e04-9d15-786de3d08e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.328077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfeed3-f80c-4e04-9d15-786de3d08e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.328292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfeed3-f80c-4e04-9d15-786de3d08e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.329016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfeed3-f80c-4e04-9d15-786de3d08e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.330634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfeed3-f80c-4e04-9d15-786de3d08e89 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.337816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 797/1588] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.338780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edea741-54f1-4843-9912-d7e3a1d1d1e6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.344495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 792/1589] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.345745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfeed3-f80c-4e04-9d15-786de3d08e89 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.355755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edea741-54f1-4843-9912-d7e3a1d1d1e6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.356095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfeed3-f80c-4e04-9d15-786de3d08e89 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.360811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7419ef0-7188-4aff-b0b4-7a81bbb7e153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.361203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003ea1-c273-41a5-8876-19316c804e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.362157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7419ef0-7188-4aff-b0b4-7a81bbb7e153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.362283 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003ea1-c273-41a5-8876-19316c804e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.362384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7419ef0-7188-4aff-b0b4-7a81bbb7e153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.362487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7419ef0-7188-4aff-b0b4-7a81bbb7e153 admin admin] PATH_INFO: `/v3/projects/4125ba5fb9944574a1d31f4f45c4dbd0/users/ba34270bf2fa4588937b7f9a22ee9c7d/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.362584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003ea1-c273-41a5-8876-19316c804e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.362964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9003ea1-c273-41a5-8876-19316c804e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.363885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003ea1-c273-41a5-8876-19316c804e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.365636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9003ea1-c273-41a5-8876-19316c804e13 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.381109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 793/1590] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.381401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7419ef0-7188-4aff-b0b4-7a81bbb7e153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4125ba5fb9944574a1d31f4f45c4dbd0, user_id=ba34270bf2fa4588937b7f9a22ee9c7d, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.382667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003ea1-c273-41a5-8876-19316c804e13 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.385190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7419ef0-7188-4aff-b0b4-7a81bbb7e153 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.396416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9003ea1-c273-41a5-8876-19316c804e13 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.412886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 798/1591] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:07 2022] PUT /identity/v3/projects/4125ba5fb9944574a1d31f4f45c4dbd0/users/ba34270bf2fa4588937b7f9a22ee9c7d/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:07.412886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7419ef0-7188-4aff-b0b4-7a81bbb7e153 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.476320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241b0f9-bfbd-469e-a20e-c992421b2032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.477472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241b0f9-bfbd-469e-a20e-c992421b2032 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.477804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241b0f9-bfbd-469e-a20e-c992421b2032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.478304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241b0f9-bfbd-469e-a20e-c992421b2032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.479064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241b0f9-bfbd-469e-a20e-c992421b2032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.480860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241b0f9-bfbd-469e-a20e-c992421b2032 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.493534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 794/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.494494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241b0f9-bfbd-469e-a20e-c992421b2032 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.504308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7419ef0-7188-4aff-b0b4-7a81bbb7e153 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.507668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241b0f9-bfbd-469e-a20e-c992421b2032 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.511212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ab829-8624-417a-b237-01c9b8d82cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.512394 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab829-8624-417a-b237-01c9b8d82cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.512674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab829-8624-417a-b237-01c9b8d82cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.512957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ab829-8624-417a-b237-01c9b8d82cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.513696 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ab829-8624-417a-b237-01c9b8d82cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.515371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ab829-8624-417a-b237-01c9b8d82cd6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.518134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9cfe-212d-4e69-8a5c-885c49c71214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.518997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9cfe-212d-4e69-8a5c-885c49c71214 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.519483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9cfe-212d-4e69-8a5c-885c49c71214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.520089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9cfe-212d-4e69-8a5c-885c49c71214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.521048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9cfe-212d-4e69-8a5c-885c49c71214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.522843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9cfe-212d-4e69-8a5c-885c49c71214 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.526761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 799/1593] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.527979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ab829-8624-417a-b237-01c9b8d82cd6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.538838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 795/1594] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.538838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9cfe-212d-4e69-8a5c-885c49c71214 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.543616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ab829-8624-417a-b237-01c9b8d82cd6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.547082 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9cfe-212d-4e69-8a5c-885c49c71214 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.550190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dbdfc-8b7d-44bf-8951-1a451654ddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.551224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dbdfc-8b7d-44bf-8951-1a451654ddf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.552119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dbdfc-8b7d-44bf-8951-1a451654ddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.552247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea463b35-3fb9-452e-8a40-2e46206a85b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.552456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dbdfc-8b7d-44bf-8951-1a451654ddf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.552815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea463b35-3fb9-452e-8a40-2e46206a85b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.553080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea463b35-3fb9-452e-8a40-2e46206a85b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.553188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dbdfc-8b7d-44bf-8951-1a451654ddf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.553350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea463b35-3fb9-452e-8a40-2e46206a85b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.554077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea463b35-3fb9-452e-8a40-2e46206a85b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.554720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dbdfc-8b7d-44bf-8951-1a451654ddf0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.555804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea463b35-3fb9-452e-8a40-2e46206a85b5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.566241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 800/1595] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.567786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 796/1596] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.568672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dbdfc-8b7d-44bf-8951-1a451654ddf0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.572579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aa3d0-4274-495e-a1d1-6cd683aefe95 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.573339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aa3d0-4274-495e-a1d1-6cd683aefe95 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.573339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aa3d0-4274-495e-a1d1-6cd683aefe95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.582315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dbdfc-8b7d-44bf-8951-1a451654ddf0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.588270 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260fab0-fbf7-4504-86b5-5ce05efd1a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.589178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260fab0-fbf7-4504-86b5-5ce05efd1a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.589466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260fab0-fbf7-4504-86b5-5ce05efd1a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.589745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260fab0-fbf7-4504-86b5-5ce05efd1a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.590650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260fab0-fbf7-4504-86b5-5ce05efd1a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.592789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260fab0-fbf7-4504-86b5-5ce05efd1a3f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.603217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e54aa3d0-4274-495e-a1d1-6cd683aefe95 None None] MFA Rules not processed for user `ba34270bf2fa4588937b7f9a22ee9c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:07.604185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 801/1597] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.604942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e54aa3d0-4274-495e-a1d1-6cd683aefe95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:07.605050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260fab0-fbf7-4504-86b5-5ce05efd1a3f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.617579 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260fab0-fbf7-4504-86b5-5ce05efd1a3f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.624375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94564067-d483-45b8-b8f3-0a62ab906f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.625087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94564067-d483-45b8-b8f3-0a62ab906f25 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.625507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94564067-d483-45b8-b8f3-0a62ab906f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.625788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94564067-d483-45b8-b8f3-0a62ab906f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.626580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94564067-d483-45b8-b8f3-0a62ab906f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.628377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94564067-d483-45b8-b8f3-0a62ab906f25 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.633956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:07.643906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 802/1598] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.644976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94564067-d483-45b8-b8f3-0a62ab906f25 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.655589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94564067-d483-45b8-b8f3-0a62ab906f25 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.671454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31968fee-c326-4525-a803-91dfb7e21e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.672524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31968fee-c326-4525-a803-91dfb7e21e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.673312 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31968fee-c326-4525-a803-91dfb7e21e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.673917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31968fee-c326-4525-a803-91dfb7e21e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.674752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31968fee-c326-4525-a803-91dfb7e21e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.677013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31968fee-c326-4525-a803-91dfb7e21e4f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.693085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 803/1599] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:07.694013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31968fee-c326-4525-a803-91dfb7e21e4f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.699424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 797/1600] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:07.709593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31968fee-c326-4525-a803-91dfb7e21e4f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.712048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54aa3d0-4274-495e-a1d1-6cd683aefe95 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.714899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee0345-3fed-4a7d-a670-ede033752d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.715934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee0345-3fed-4a7d-a670-ede033752d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.716287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee0345-3fed-4a7d-a670-ede033752d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.716643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee0345-3fed-4a7d-a670-ede033752d4b admin admin] PATH_INFO: `/v3/projects/33eaf69c5a8f434a98fb0940ad8464bf/users/fdb0aef8e7ff4e6089d4ea780a85b0fe/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.727563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54aa3d0-4274-495e-a1d1-6cd683aefe95 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.735883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e76e3-2930-4b1c-9884-cc302c4a753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.736923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee0345-3fed-4a7d-a670-ede033752d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=33eaf69c5a8f434a98fb0940ad8464bf, user_id=fdb0aef8e7ff4e6089d4ea780a85b0fe, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.737297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e76e3-2930-4b1c-9884-cc302c4a753e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.738805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e76e3-2930-4b1c-9884-cc302c4a753e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.739247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e76e3-2930-4b1c-9884-cc302c4a753e admin admin] PATH_INFO: `/v3/projects/b44eff44cb504e1f9bf037303db24d1b/users/868e2f0e822b430981369d414eee8e53/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.740744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee0345-3fed-4a7d-a670-ede033752d4b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.757518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e76e3-2930-4b1c-9884-cc302c4a753e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44eff44cb504e1f9bf037303db24d1b, user_id=868e2f0e822b430981369d414eee8e53, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.776586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e76e3-2930-4b1c-9884-cc302c4a753e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.778148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 804/1601] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:07 2022] PUT /identity/v3/projects/33eaf69c5a8f434a98fb0940ad8464bf/users/fdb0aef8e7ff4e6089d4ea780a85b0fe/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:07.778148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee0345-3fed-4a7d-a670-ede033752d4b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.797622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 798/1602] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:07 2022] PUT /identity/v3/projects/b44eff44cb504e1f9bf037303db24d1b/users/868e2f0e822b430981369d414eee8e53/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:07.801337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e76e3-2930-4b1c-9884-cc302c4a753e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.833842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:07.856213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee0345-3fed-4a7d-a670-ede033752d4b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.867946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e76e3-2930-4b1c-9884-cc302c4a753e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:07.876012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73df2e0-f5e6-483d-b868-ca44871a2779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.877391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73df2e0-f5e6-483d-b868-ca44871a2779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.878729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73df2e0-f5e6-483d-b868-ca44871a2779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.879258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73df2e0-f5e6-483d-b868-ca44871a2779 admin admin] PATH_INFO: `/v3/projects/7f39e852f8d14dec8e98d8d77fe90766/users/112d731047a64efc8e9143c4d471a7d6/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.898224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73df2e0-f5e6-483d-b868-ca44871a2779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f39e852f8d14dec8e98d8d77fe90766, user_id=112d731047a64efc8e9143c4d471a7d6, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.901775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73df2e0-f5e6-483d-b868-ca44871a2779 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.917265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 799/1603] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:07 2022] PUT /identity/v3/projects/7f39e852f8d14dec8e98d8d77fe90766/users/112d731047a64efc8e9143c4d471a7d6/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:07.918403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73df2e0-f5e6-483d-b868-ca44871a2779 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.921937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2603d-87ec-4a87-a054-d0f1a17f2df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:07.922924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2603d-87ec-4a87-a054-d0f1a17f2df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:07.923280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2603d-87ec-4a87-a054-d0f1a17f2df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:07.923654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2603d-87ec-4a87-a054-d0f1a17f2df0 admin admin] PATH_INFO: `/v3/projects/53c88ba2f03f47538ca0eaeb08ccfc07/users/c941dc00e0584666aba13263b8c1fcb8/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:07.940422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2603d-87ec-4a87-a054-d0f1a17f2df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c88ba2f03f47538ca0eaeb08ccfc07, user_id=c941dc00e0584666aba13263b8c1fcb8, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:07.943594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2603d-87ec-4a87-a054-d0f1a17f2df0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:07.960012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 805/1604] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:07 2022] PUT /identity/v3/projects/53c88ba2f03f47538ca0eaeb08ccfc07/users/c941dc00e0584666aba13263b8c1fcb8/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:07.960853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2603d-87ec-4a87-a054-d0f1a17f2df0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:07.993070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73df2e0-f5e6-483d-b868-ca44871a2779 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.027573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2603d-87ec-4a87-a054-d0f1a17f2df0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.029811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:08.032665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c4b7f-9d79-4416-9e30-f9ddc6849a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.033713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c4b7f-9d79-4416-9e30-f9ddc6849a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.034112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c4b7f-9d79-4416-9e30-f9ddc6849a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.034112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c4b7f-9d79-4416-9e30-f9ddc6849a39 admin admin] PATH_INFO: `/v3/projects/e24dd4aa9a164db392143ccd6996df8a/users/5de4fc94af07472898998a209b87e46f/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.051888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c4b7f-9d79-4416-9e30-f9ddc6849a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24dd4aa9a164db392143ccd6996df8a, user_id=5de4fc94af07472898998a209b87e46f, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.055750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c4b7f-9d79-4416-9e30-f9ddc6849a39 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.073815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e494154-0163-4fcf-af8e-ddb0d495f14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.074905 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e494154-0163-4fcf-af8e-ddb0d495f14a admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.075325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e494154-0163-4fcf-af8e-ddb0d495f14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.075732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e494154-0163-4fcf-af8e-ddb0d495f14a admin admin] PATH_INFO: `/v3/projects/6aae4119457442c2849db1540801d3d8/users/84ba1f7871004be782bc51daf1235c41/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.077763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 806/1605] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:07 2022] PUT /identity/v3/projects/e24dd4aa9a164db392143ccd6996df8a/users/5de4fc94af07472898998a209b87e46f/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:08.080556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c4b7f-9d79-4416-9e30-f9ddc6849a39 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.093147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e494154-0163-4fcf-af8e-ddb0d495f14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aae4119457442c2849db1540801d3d8, user_id=84ba1f7871004be782bc51daf1235c41, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.096051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e494154-0163-4fcf-af8e-ddb0d495f14a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.122206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 800/1606] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:07 2022] PUT /identity/v3/projects/6aae4119457442c2849db1540801d3d8/users/84ba1f7871004be782bc51daf1235c41/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:08.122206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2723-8ee2-4136-afc8-7b2dc0139dda None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.122206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2723-8ee2-4136-afc8-7b2dc0139dda None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.122206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2723-8ee2-4136-afc8-7b2dc0139dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.158143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:08.161579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-817f2723-8ee2-4136-afc8-7b2dc0139dda None None] MFA Rules not processed for user `fdb0aef8e7ff4e6089d4ea780a85b0fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:08.162959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-817f2723-8ee2-4136-afc8-7b2dc0139dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:08.164031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c4b7f-9d79-4416-9e30-f9ddc6849a39 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.240755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a8921-8b86-4876-a07c-c4790fd711b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.241984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a8921-8b86-4876-a07c-c4790fd711b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.241984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a8921-8b86-4876-a07c-c4790fd711b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.242248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a8921-8b86-4876-a07c-c4790fd711b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.242875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a8921-8b86-4876-a07c-c4790fd711b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.245977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a8921-8b86-4876-a07c-c4790fd711b5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.250879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 801/1607] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:08.252406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04041626-202f-4e7d-a92c-8a2fc9500c5b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.252792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04041626-202f-4e7d-a92c-8a2fc9500c5b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.253170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04041626-202f-4e7d-a92c-8a2fc9500c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.276352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-04041626-202f-4e7d-a92c-8a2fc9500c5b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:08.277698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-04041626-202f-4e7d-a92c-8a2fc9500c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:08.285546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 807/1608] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:08.287150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e8835-5738-4d5d-95a6-5135b788e8ab None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.287475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e8835-5738-4d5d-95a6-5135b788e8ab None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.287709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 802/1609] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:08.289084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478198e2-1f7c-44fd-a8c5-b9d01a6c7803 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.289448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478198e2-1f7c-44fd-a8c5-b9d01a6c7803 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.289845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478198e2-1f7c-44fd-a8c5-b9d01a6c7803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.297781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e8835-5738-4d5d-95a6-5135b788e8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.314076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-478198e2-1f7c-44fd-a8c5-b9d01a6c7803 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:08.315566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-478198e2-1f7c-44fd-a8c5-b9d01a6c7803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:08.317931 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c55e8835-5738-4d5d-95a6-5135b788e8ab None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:08.319095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c55e8835-5738-4d5d-95a6-5135b788e8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:08.393148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 808/1610] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:08.398565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3096-eb8e-4bad-9ba5-4e1c31e993ab None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.398867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3096-eb8e-4bad-9ba5-4e1c31e993ab None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.399214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c3096-eb8e-4bad-9ba5-4e1c31e993ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.403503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 803/1611] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:08.403503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc590548-011f-4886-99fb-6faa12f4323e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.403503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc590548-011f-4886-99fb-6faa12f4323e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.403503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc590548-011f-4886-99fb-6faa12f4323e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.422534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dc590548-011f-4886-99fb-6faa12f4323e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:08.423968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dc590548-011f-4886-99fb-6faa12f4323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:08.434438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 804/1612] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:08.435270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc590548-011f-4886-99fb-6faa12f4323e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.440535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-070c3096-eb8e-4bad-9ba5-4e1c31e993ab None None] MFA Rules not processed for user `5de4fc94af07472898998a209b87e46f`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:08.441771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-070c3096-eb8e-4bad-9ba5-4e1c31e993ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:08.454858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc590548-011f-4886-99fb-6faa12f4323e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.463030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183ff75-c5b0-43ab-80d2-37b9868dd225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.463985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183ff75-c5b0-43ab-80d2-37b9868dd225 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.464410 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183ff75-c5b0-43ab-80d2-37b9868dd225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.464841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183ff75-c5b0-43ab-80d2-37b9868dd225 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.465706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183ff75-c5b0-43ab-80d2-37b9868dd225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.467814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183ff75-c5b0-43ab-80d2-37b9868dd225 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.509412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 805/1613] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/projects => generated 357 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:08.510943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183ff75-c5b0-43ab-80d2-37b9868dd225 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.524835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:08.534468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 809/1614] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:08.534468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c3096-eb8e-4bad-9ba5-4e1c31e993ab None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.564708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:08.613121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c3096-eb8e-4bad-9ba5-4e1c31e993ab None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.614599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183ff75-c5b0-43ab-80d2-37b9868dd225 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.621531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e2ee7-95e5-4071-b370-e3afcb9ff4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.625332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e2ee7-95e5-4071-b370-e3afcb9ff4bd admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.625961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e2ee7-95e5-4071-b370-e3afcb9ff4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.626247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e2ee7-95e5-4071-b370-e3afcb9ff4bd admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.626998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e2ee7-95e5-4071-b370-e3afcb9ff4bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.629048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e2ee7-95e5-4071-b370-e3afcb9ff4bd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.660886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 806/1615] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:08.662112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e2ee7-95e5-4071-b370-e3afcb9ff4bd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.682768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a436f-1b2c-4013-903c-fbb69889084a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.683728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a436f-1b2c-4013-903c-fbb69889084a service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.684138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a436f-1b2c-4013-903c-fbb69889084a service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.684525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a436f-1b2c-4013-903c-fbb69889084a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.696409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a436f-1b2c-4013-903c-fbb69889084a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.698102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a436f-1b2c-4013-903c-fbb69889084a service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.709495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 810/1616] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:08.710397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a436f-1b2c-4013-903c-fbb69889084a service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.719462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a436f-1b2c-4013-903c-fbb69889084a service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.723443 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844ce8c-f838-4d9f-b9c7-ef5ea8c27a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.724458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844ce8c-f838-4d9f-b9c7-ef5ea8c27a71 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.724972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844ce8c-f838-4d9f-b9c7-ef5ea8c27a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.725355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844ce8c-f838-4d9f-b9c7-ef5ea8c27a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.735813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844ce8c-f838-4d9f-b9c7-ef5ea8c27a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.737431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844ce8c-f838-4d9f-b9c7-ef5ea8c27a71 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.748866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 811/1617] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:08.749914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844ce8c-f838-4d9f-b9c7-ef5ea8c27a71 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.760433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844ce8c-f838-4d9f-b9c7-ef5ea8c27a71 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.765584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c11618-4e67-4db9-8130-177f66d69dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.766397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c11618-4e67-4db9-8130-177f66d69dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.766839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c11618-4e67-4db9-8130-177f66d69dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.767388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c11618-4e67-4db9-8130-177f66d69dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.781322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e2ee7-95e5-4071-b370-e3afcb9ff4bd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.782205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c11618-4e67-4db9-8130-177f66d69dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.783467 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c11618-4e67-4db9-8130-177f66d69dd3 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.791217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e962b87-641f-47d7-aa79-5246d0273072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.792486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e962b87-641f-47d7-aa79-5246d0273072 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.793254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e962b87-641f-47d7-aa79-5246d0273072 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.793254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e962b87-641f-47d7-aa79-5246d0273072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.799991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 812/1618] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:08.800999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c11618-4e67-4db9-8130-177f66d69dd3 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.814493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c11618-4e67-4db9-8130-177f66d69dd3 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.816900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e962b87-641f-47d7-aa79-5246d0273072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.818360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e962b87-641f-47d7-aa79-5246d0273072 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.822598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f4760-7577-43fe-812f-5a32e31f87a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.823494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f4760-7577-43fe-812f-5a32e31f87a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.823923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f4760-7577-43fe-812f-5a32e31f87a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.824304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f4760-7577-43fe-812f-5a32e31f87a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.825196 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f4760-7577-43fe-812f-5a32e31f87a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.827784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f4760-7577-43fe-812f-5a32e31f87a2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.834380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 807/1619] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:08.835872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e962b87-641f-47d7-aa79-5246d0273072 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.848715 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e962b87-641f-47d7-aa79-5246d0273072 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.864630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 813/1620] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/users => generated 285 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:08.865723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f4760-7577-43fe-812f-5a32e31f87a2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.874944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3bb39-5bff-4468-b6d1-438900e09f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.877377 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3bb39-5bff-4468-b6d1-438900e09f12 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.877864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3bb39-5bff-4468-b6d1-438900e09f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.878330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3bb39-5bff-4468-b6d1-438900e09f12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.879258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f4760-7577-43fe-812f-5a32e31f87a2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:08.879428 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3bb39-5bff-4468-b6d1-438900e09f12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.881612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3bb39-5bff-4468-b6d1-438900e09f12 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.886164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bdbf6-68cb-4ca2-95b8-bdb6f18e0e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:08.887737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bdbf6-68cb-4ca2-95b8-bdb6f18e0e26 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:08.888318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bdbf6-68cb-4ca2-95b8-bdb6f18e0e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:08.888996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bdbf6-68cb-4ca2-95b8-bdb6f18e0e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:08.890127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bdbf6-68cb-4ca2-95b8-bdb6f18e0e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:08.893103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bdbf6-68cb-4ca2-95b8-bdb6f18e0e26 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:08.913666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 814/1621] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:08.916805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bdbf6-68cb-4ca2-95b8-bdb6f18e0e26 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.939584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 808/1622] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:08 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:08.941435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3bb39-5bff-4468-b6d1-438900e09f12 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:08.943111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bdbf6-68cb-4ca2-95b8-bdb6f18e0e26 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.057949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3bb39-5bff-4468-b6d1-438900e09f12 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.064032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b8df1-f942-4acc-b9d8-6f0a7503efb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.064760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b8df1-f942-4acc-b9d8-6f0a7503efb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.064881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e31f45-3abc-421a-ae87-1f3befe65747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.065052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b8df1-f942-4acc-b9d8-6f0a7503efb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.065362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b8df1-f942-4acc-b9d8-6f0a7503efb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.065967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31f45-3abc-421a-ae87-1f3befe65747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.066311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31f45-3abc-421a-ae87-1f3befe65747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.066466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b8df1-f942-4acc-b9d8-6f0a7503efb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.066597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31f45-3abc-421a-ae87-1f3befe65747 admin admin] PATH_INFO: `/v3/projects/b62abe52aef048f597242629585ed5b2/users/4de91929e4b64a19bb4cee7ecb39752f/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.068630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b8df1-f942-4acc-b9d8-6f0a7503efb3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.087733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e31f45-3abc-421a-ae87-1f3befe65747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62abe52aef048f597242629585ed5b2, user_id=4de91929e4b64a19bb4cee7ecb39752f, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.090479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e31f45-3abc-421a-ae87-1f3befe65747 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.096049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 815/1623] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:09.098251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b8df1-f942-4acc-b9d8-6f0a7503efb3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:09.106122 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:09.112963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 809/1624] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:08 2022] PUT /identity/v3/projects/b62abe52aef048f597242629585ed5b2/users/4de91929e4b64a19bb4cee7ecb39752f/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:09.114311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e31f45-3abc-421a-ae87-1f3befe65747 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:09.121284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b8df1-f942-4acc-b9d8-6f0a7503efb3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.177305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:09.198070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e31f45-3abc-421a-ae87-1f3befe65747 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.203064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e781ffe-067d-49ec-afab-fdd37b3303f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.215806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e781ffe-067d-49ec-afab-fdd37b3303f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.311582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e781ffe-067d-49ec-afab-fdd37b3303f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.311582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e781ffe-067d-49ec-afab-fdd37b3303f9 admin admin] PATH_INFO: `/v3/projects/24f7faf54dd447b1a8ed0b8ada9004c4/users/0bceddefdd3d4b59a03ca15973c84e48/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.311582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e781ffe-067d-49ec-afab-fdd37b3303f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f7faf54dd447b1a8ed0b8ada9004c4, user_id=0bceddefdd3d4b59a03ca15973c84e48, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.311582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e781ffe-067d-49ec-afab-fdd37b3303f9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.311582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 810/1625] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:09 2022] PUT /identity/v3/projects/24f7faf54dd447b1a8ed0b8ada9004c4/users/0bceddefdd3d4b59a03ca15973c84e48/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:09.313076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e781ffe-067d-49ec-afab-fdd37b3303f9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:09.366392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee4a22-35c6-49fc-b761-998cea1e7b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.367245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee4a22-35c6-49fc-b761-998cea1e7b3a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.367558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee4a22-35c6-49fc-b761-998cea1e7b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.367982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee4a22-35c6-49fc-b761-998cea1e7b3a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.368787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee4a22-35c6-49fc-b761-998cea1e7b3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.370717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee4a22-35c6-49fc-b761-998cea1e7b3a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.419925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 816/1626] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:09 2022] POST /identity/v3/users => generated 308 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:09.420463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee4a22-35c6-49fc-b761-998cea1e7b3a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:09.456141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee4a22-35c6-49fc-b761-998cea1e7b3a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.488661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e781ffe-067d-49ec-afab-fdd37b3303f9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.506631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5238b-5646-43e2-92b5-4622bab6e952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.507373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5238b-5646-43e2-92b5-4622bab6e952 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.507679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5238b-5646-43e2-92b5-4622bab6e952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.507935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5238b-5646-43e2-92b5-4622bab6e952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.508955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5238b-5646-43e2-92b5-4622bab6e952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.512527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad5238b-5646-43e2-92b5-4622bab6e952 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.536139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 811/1627] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:09.536693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5238b-5646-43e2-92b5-4622bab6e952 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:09.570531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fcfb4-7112-420b-8624-c4106f753579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.572135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fcfb4-7112-420b-8624-c4106f753579 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.572769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fcfb4-7112-420b-8624-c4106f753579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.573283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fcfb4-7112-420b-8624-c4106f753579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.574411 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fcfb4-7112-420b-8624-c4106f753579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.577561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad5238b-5646-43e2-92b5-4622bab6e952 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.577753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fcfb4-7112-420b-8624-c4106f753579 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.595280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 817/1628] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:09.595280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fcfb4-7112-420b-8624-c4106f753579 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:09.597243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09adcd-ece4-4586-922c-1bfe3c01e374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.599603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09adcd-ece4-4586-922c-1bfe3c01e374 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.600078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09adcd-ece4-4586-922c-1bfe3c01e374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.601896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09adcd-ece4-4586-922c-1bfe3c01e374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.606401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09adcd-ece4-4586-922c-1bfe3c01e374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.607533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fcfb4-7112-420b-8624-c4106f753579 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.608589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09adcd-ece4-4586-922c-1bfe3c01e374 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.614898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fa0a1-f547-4138-8c58-a8c8c012bb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.616512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fa0a1-f547-4138-8c58-a8c8c012bb4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.617199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fa0a1-f547-4138-8c58-a8c8c012bb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.617824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056fa0a1-f547-4138-8c58-a8c8c012bb4c admin admin] PATH_INFO: `/v3/projects/b62abe52aef048f597242629585ed5b2/users/4de91929e4b64a19bb4cee7ecb39752f/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.620600 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fa0a1-f547-4138-8c58-a8c8c012bb4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62abe52aef048f597242629585ed5b2, user_id=4de91929e4b64a19bb4cee7ecb39752f, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.624295 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056fa0a1-f547-4138-8c58-a8c8c012bb4c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.640194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 812/1629] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:09.642725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09adcd-ece4-4586-922c-1bfe3c01e374 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:09.672059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 818/1630] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:09 2022] PUT /identity/v3/projects/b62abe52aef048f597242629585ed5b2/users/4de91929e4b64a19bb4cee7ecb39752f/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:09.674619 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fa0a1-f547-4138-8c58-a8c8c012bb4c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:09.675141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09adcd-ece4-4586-922c-1bfe3c01e374 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.790398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea28c9c-f4be-45b9-86b1-3744c6e3c739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.792286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea28c9c-f4be-45b9-86b1-3744c6e3c739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.792835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea28c9c-f4be-45b9-86b1-3744c6e3c739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.792964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056fa0a1-f547-4138-8c58-a8c8c012bb4c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:09.793536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea28c9c-f4be-45b9-86b1-3744c6e3c739 admin admin] PATH_INFO: `/v3/projects/24f7faf54dd447b1a8ed0b8ada9004c4/users/0bceddefdd3d4b59a03ca15973c84e48/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.796387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea28c9c-f4be-45b9-86b1-3744c6e3c739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f7faf54dd447b1a8ed0b8ada9004c4, user_id=0bceddefdd3d4b59a03ca15973c84e48, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.800074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea28c9c-f4be-45b9-86b1-3744c6e3c739 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.801796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909a4da-168d-4d2d-9577-e815e40732ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:09.802768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909a4da-168d-4d2d-9577-e815e40732ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.803058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909a4da-168d-4d2d-9577-e815e40732ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.803354 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909a4da-168d-4d2d-9577-e815e40732ec admin admin] PATH_INFO: `/v3/projects/eaa7b24c9e98491da5f783cb50f5a638/users/6f1695ef7e3d49fe9943b14d88e03ff9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.815973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 813/1631] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:09 2022] PUT /identity/v3/projects/24f7faf54dd447b1a8ed0b8ada9004c4/users/0bceddefdd3d4b59a03ca15973c84e48/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:09.824362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5a2a-d4e0-4472-9305-6d0cc230eb69 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.824744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5a2a-d4e0-4472-9305-6d0cc230eb69 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.826614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5a2a-d4e0-4472-9305-6d0cc230eb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.839045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909a4da-168d-4d2d-9577-e815e40732ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa7b24c9e98491da5f783cb50f5a638, user_id=6f1695ef7e3d49fe9943b14d88e03ff9, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:09.842183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909a4da-168d-4d2d-9577-e815e40732ec admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:09.844015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:09.876874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 819/1632] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:09 2022] PUT /identity/v3/projects/eaa7b24c9e98491da5f783cb50f5a638/users/6f1695ef7e3d49fe9943b14d88e03ff9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:09.878609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fba7d-c4d0-4adf-84db-a24a3cc6d332 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:09.878907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fba7d-c4d0-4adf-84db-a24a3cc6d332 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:09.879188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fba7d-c4d0-4adf-84db-a24a3cc6d332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:09.885312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d49e5a2a-d4e0-4472-9305-6d0cc230eb69 None None] MFA Rules not processed for user `4de91929e4b64a19bb4cee7ecb39752f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:09.887098 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d49e5a2a-d4e0-4472-9305-6d0cc230eb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:09.930330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:09.935150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4a8fba7d-c4d0-4adf-84db-a24a3cc6d332 None None] MFA Rules not processed for user `0bceddefdd3d4b59a03ca15973c84e48`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:09.937561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4a8fba7d-c4d0-4adf-84db-a24a3cc6d332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:10.070082 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 814/1633] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:10.072556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e5a2a-d4e0-4472-9305-6d0cc230eb69 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:10.137972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 820/1634] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:09 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:10.146156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fba7d-c4d0-4adf-84db-a24a3cc6d332 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:10.236994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e5a2a-d4e0-4472-9305-6d0cc230eb69 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:10.248706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1ca9f-f64d-435f-9e0f-2668783cffc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:10.249496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1ca9f-f64d-435f-9e0f-2668783cffc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:10.252172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1ca9f-f64d-435f-9e0f-2668783cffc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:10.252542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1ca9f-f64d-435f-9e0f-2668783cffc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:10.253777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1ca9f-f64d-435f-9e0f-2668783cffc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:10.261034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1ca9f-f64d-435f-9e0f-2668783cffc9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:10.277743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 815/1635] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:10.278757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1ca9f-f64d-435f-9e0f-2668783cffc9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:10.296343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fba7d-c4d0-4adf-84db-a24a3cc6d332 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:10.306763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1ca9f-f64d-435f-9e0f-2668783cffc9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:10.310894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efbb44-3f01-4e14-bd2e-d84ca3d7e2ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:10.312353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efbb44-3f01-4e14-bd2e-d84ca3d7e2ff service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:10.312980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efbb44-3f01-4e14-bd2e-d84ca3d7e2ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:10.314185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efbb44-3f01-4e14-bd2e-d84ca3d7e2ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:10.315844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790aa4e-e2a7-4350-a0e5-5d4b3455d55d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:10.316724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790aa4e-e2a7-4350-a0e5-5d4b3455d55d service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:10.317012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790aa4e-e2a7-4350-a0e5-5d4b3455d55d service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:10.318823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790aa4e-e2a7-4350-a0e5-5d4b3455d55d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:10.335714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790aa4e-e2a7-4350-a0e5-5d4b3455d55d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:10.337135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790aa4e-e2a7-4350-a0e5-5d4b3455d55d service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:10.340091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efbb44-3f01-4e14-bd2e-d84ca3d7e2ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:10.342632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efbb44-3f01-4e14-bd2e-d84ca3d7e2ff service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:10.361994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 816/1636] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:10 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:10.363867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790aa4e-e2a7-4350-a0e5-5d4b3455d55d service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:10.369733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 821/1637] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:10.421832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790aa4e-e2a7-4350-a0e5-5d4b3455d55d service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:10.434986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea0f17-3cd7-41eb-b851-648d29d113ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:10.436918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea0f17-3cd7-41eb-b851-648d29d113ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:10.437864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea0f17-3cd7-41eb-b851-648d29d113ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:10.437864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea0f17-3cd7-41eb-b851-648d29d113ee admin admin] PATH_INFO: `/v3/projects/eaa7b24c9e98491da5f783cb50f5a638/users/6f1695ef7e3d49fe9943b14d88e03ff9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:10.446537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea0f17-3cd7-41eb-b851-648d29d113ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa7b24c9e98491da5f783cb50f5a638, user_id=6f1695ef7e3d49fe9943b14d88e03ff9, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:10.452788 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea0f17-3cd7-41eb-b851-648d29d113ee admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:10.464817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 817/1638] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:10 2022] PUT /identity/v3/projects/eaa7b24c9e98491da5f783cb50f5a638/users/6f1695ef7e3d49fe9943b14d88e03ff9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:10.475582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f30985-c62e-483e-bb40-7529ab613735 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:10.476499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f30985-c62e-483e-bb40-7529ab613735 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:10.476499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f30985-c62e-483e-bb40-7529ab613735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:10.529548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e4f30985-c62e-483e-bb40-7529ab613735 None None] MFA Rules not processed for user `6f1695ef7e3d49fe9943b14d88e03ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:10.531208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e4f30985-c62e-483e-bb40-7529ab613735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:10.550917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea0f17-3cd7-41eb-b851-648d29d113ee admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:10.685525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea0f17-3cd7-41eb-b851-648d29d113ee admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:10.695692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d977f67-35ee-4ae4-bbf9-35f88305a54e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:10.696170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d977f67-35ee-4ae4-bbf9-35f88305a54e service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:10.696469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d977f67-35ee-4ae4-bbf9-35f88305a54e service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:10.696687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d977f67-35ee-4ae4-bbf9-35f88305a54e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:10.703970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 822/1639] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:10 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:10.705244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3a937-0947-4704-a2ab-e4359bd0d0d5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:10.705533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3a937-0947-4704-a2ab-e4359bd0d0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:10.705813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3a937-0947-4704-a2ab-e4359bd0d0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:10.716331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:10.776418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d977f67-35ee-4ae4-bbf9-35f88305a54e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:10.777985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d977f67-35ee-4ae4-bbf9-35f88305a54e service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:10.779202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-39b3a937-0947-4704-a2ab-e4359bd0d0d5 None None] MFA Rules not processed for user `84ba1f7871004be782bc51daf1235c41`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:10.780752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-39b3a937-0947-4704-a2ab-e4359bd0d0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:10.789322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 818/1640] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:10.790380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d977f67-35ee-4ae4-bbf9-35f88305a54e service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:10.806477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d977f67-35ee-4ae4-bbf9-35f88305a54e service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:10.810318 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:10.818817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c9b5e-733a-43ca-bfe3-7f079c2ffd04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:10.819518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c9b5e-733a-43ca-bfe3-7f079c2ffd04 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:10.819799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c9b5e-733a-43ca-bfe3-7f079c2ffd04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:10.820075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c9b5e-733a-43ca-bfe3-7f079c2ffd04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:10.895127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 823/1641] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:10.900385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3a937-0947-4704-a2ab-e4359bd0d0d5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:10.966844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c9b5e-733a-43ca-bfe3-7f079c2ffd04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:10.969771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939c9b5e-733a-43ca-bfe3-7f079c2ffd04 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:10.982654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 819/1642] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:10 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:11.049008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3a937-0947-4704-a2ab-e4359bd0d0d5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:11.057806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be38662-4c02-4757-819b-3ba191b690b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:11.058798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be38662-4c02-4757-819b-3ba191b690b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:11.058919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be38662-4c02-4757-819b-3ba191b690b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:11.061082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be38662-4c02-4757-819b-3ba191b690b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:11.065119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be38662-4c02-4757-819b-3ba191b690b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:11.073407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be38662-4c02-4757-819b-3ba191b690b7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:11.153435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 824/1643] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:10 2022] POST /identity/v3/projects => generated 405 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:11.155117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c9b5e-733a-43ca-bfe3-7f079c2ffd04 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:11.174951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939c9b5e-733a-43ca-bfe3-7f079c2ffd04 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:11.210866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be38662-4c02-4757-819b-3ba191b690b7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:11.369718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04beed-c3eb-43cc-af24-793630161482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:11.375551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04beed-c3eb-43cc-af24-793630161482 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:11.375891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04beed-c3eb-43cc-af24-793630161482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:11.376120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04beed-c3eb-43cc-af24-793630161482 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:11.376838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04beed-c3eb-43cc-af24-793630161482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:11.404980 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be38662-4c02-4757-819b-3ba191b690b7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:11.406658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc04beed-c3eb-43cc-af24-793630161482 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:11.422456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b76441-fcfb-4ace-b03f-ab01889f0a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:11.424803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b76441-fcfb-4ace-b03f-ab01889f0a4d service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:11.425679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b76441-fcfb-4ace-b03f-ab01889f0a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:11.426322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b76441-fcfb-4ace-b03f-ab01889f0a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:11.454516 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 820/1644] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:11 2022] POST /identity/v3/users => generated 309 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:11.461789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04beed-c3eb-43cc-af24-793630161482 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:11.490488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc04beed-c3eb-43cc-af24-793630161482 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:11.500638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a3b11-c8b5-467e-86fa-fb61f5d0c8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:11.503002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a3b11-c8b5-467e-86fa-fb61f5d0c8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:11.504076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a3b11-c8b5-467e-86fa-fb61f5d0c8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:11.504605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a3b11-c8b5-467e-86fa-fb61f5d0c8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:11.507282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a3b11-c8b5-467e-86fa-fb61f5d0c8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:11.510635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a3b11-c8b5-467e-86fa-fb61f5d0c8aa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:11.565744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:11.581320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b76441-fcfb-4ace-b03f-ab01889f0a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:11.582787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b76441-fcfb-4ace-b03f-ab01889f0a4d service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:11.599791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 821/1645] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:11.605086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a3b11-c8b5-467e-86fa-fb61f5d0c8aa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:11.606159 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 825/1646] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:11 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:11.618255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a3b11-c8b5-467e-86fa-fb61f5d0c8aa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:11.624138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbda00-e751-4a46-9132-79e98629dc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:11.625422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbda00-e751-4a46-9132-79e98629dc99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:11.626259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbda00-e751-4a46-9132-79e98629dc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:11.627111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbda00-e751-4a46-9132-79e98629dc99 admin admin] PATH_INFO: `/v3/projects/17c03a1303434dc48a16d1a5daa886d1/users/4503bfcd35aa4936902deba9f2228894/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:11.652385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbda00-e751-4a46-9132-79e98629dc99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c03a1303434dc48a16d1a5daa886d1, user_id=4503bfcd35aa4936902deba9f2228894, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:11.655806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbda00-e751-4a46-9132-79e98629dc99 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:11.706422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b76441-fcfb-4ace-b03f-ab01889f0a4d service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:11.708496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 822/1647] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:11 2022] PUT /identity/v3/projects/17c03a1303434dc48a16d1a5daa886d1/users/4503bfcd35aa4936902deba9f2228894/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:11.724085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b76441-fcfb-4ace-b03f-ab01889f0a4d service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:11.898095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed3b6a-41e6-4ba8-a941-7328ba1e3526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:11.898837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed3b6a-41e6-4ba8-a941-7328ba1e3526 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:11.899252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed3b6a-41e6-4ba8-a941-7328ba1e3526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:11.899737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed3b6a-41e6-4ba8-a941-7328ba1e3526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:11.910780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed3b6a-41e6-4ba8-a941-7328ba1e3526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:11.915578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed3b6a-41e6-4ba8-a941-7328ba1e3526 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:11.962451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbda00-e751-4a46-9132-79e98629dc99 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:11.965115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 826/1648] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:34:11.971022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:11.980866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbda00-e751-4a46-9132-79e98629dc99 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:11.998077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae1c6d-71e9-4572-b187-67ab48d42103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:11.999168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae1c6d-71e9-4572-b187-67ab48d42103 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:11.999579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae1c6d-71e9-4572-b187-67ab48d42103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:11.999862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae1c6d-71e9-4572-b187-67ab48d42103 admin admin] PATH_INFO: `/v3/projects/17c03a1303434dc48a16d1a5daa886d1/users/4503bfcd35aa4936902deba9f2228894/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:12.009727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae1c6d-71e9-4572-b187-67ab48d42103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c03a1303434dc48a16d1a5daa886d1, user_id=4503bfcd35aa4936902deba9f2228894, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:12.012559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae1c6d-71e9-4572-b187-67ab48d42103 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:12.044664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 823/1649] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:11 2022] PUT /identity/v3/projects/17c03a1303434dc48a16d1a5daa886d1/users/4503bfcd35aa4936902deba9f2228894/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:12.495719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed3b6a-41e6-4ba8-a941-7328ba1e3526 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:12.508128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed3b6a-41e6-4ba8-a941-7328ba1e3526 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:12.660014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:12.660920 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:12.661203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:12.661480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] PATH_INFO: `/v3/users/5de4fc94af07472898998a209b87e46f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:12.663984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de4fc94af07472898998a209b87e46f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:12.666148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:12.761207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de4fc94af07472898998a209b87e46f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:12.762044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de4fc94af07472898998a209b87e46f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:12.778197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de4fc94af07472898998a209b87e46f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:12.778823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de4fc94af07472898998a209b87e46f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:12.809877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de4fc94af07472898998a209b87e46f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:12.833926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 827/1650] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:12 2022] DELETE /identity/v3/users/5de4fc94af07472898998a209b87e46f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:12.838734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0877266-9c4b-4126-979f-2a150077486d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:12.839020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0877266-9c4b-4126-979f-2a150077486d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:12.839315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0877266-9c4b-4126-979f-2a150077486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:12.872097 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b0877266-9c4b-4126-979f-2a150077486d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:12.873531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b0877266-9c4b-4126-979f-2a150077486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:13.004592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 824/1651] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:13.040174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:13.041656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:13.056298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:13.164546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e208f29-b2cf-4136-9648-d0019c5adb32 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:13.178903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204b4ce-f643-4679-84c7-980223dc8c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:13.179767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204b4ce-f643-4679-84c7-980223dc8c57 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:13.179992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204b4ce-f643-4679-84c7-980223dc8c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:13.180215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204b4ce-f643-4679-84c7-980223dc8c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:13.198791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204b4ce-f643-4679-84c7-980223dc8c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:13.200486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204b4ce-f643-4679-84c7-980223dc8c57 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:13.230763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 828/1652] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:14.262247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2e9c5-cac6-44f8-8c94-c6d9a958e706 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:14.262722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2e9c5-cac6-44f8-8c94-c6d9a958e706 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:14.262975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2e9c5-cac6-44f8-8c94-c6d9a958e706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:14.328718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b3d2e9c5-cac6-44f8-8c94-c6d9a958e706 None None] MFA Rules not processed for user `4503bfcd35aa4936902deba9f2228894`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:14.334407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b3d2e9c5-cac6-44f8-8c94-c6d9a958e706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:14.533468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 825/1653] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:14.933839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204b4ce-f643-4679-84c7-980223dc8c57 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:14.935429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2204b4ce-f643-4679-84c7-980223dc8c57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:14.957173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204b4ce-f643-4679-84c7-980223dc8c57 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:14.977262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:14.978353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:14.978650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:14.978926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] PATH_INFO: `/v3/projects/e24dd4aa9a164db392143ccd6996df8a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:14.981340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24dd4aa9a164db392143ccd6996df8a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:14.982380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: WARNING py.warnings [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:34:14.982380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: warnings.warn(msg) Jul 02 03:34:14.982380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: Jul 02 03:34:14.987836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:15.146184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24dd4aa9a164db392143ccd6996df8a', 'request_id': 'req-858485cc-052a-4441-8efd-08362893de07'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.160584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24dd4aa9a164db392143ccd6996df8a', 'request_id': 'req-858485cc-052a-4441-8efd-08362893de07'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.180461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24dd4aa9a164db392143ccd6996df8a', 'request_id': 'req-858485cc-052a-4441-8efd-08362893de07'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.202597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] The token cache is being invalidate because project e24dd4aa9a164db392143ccd6996df8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:15.202912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.207858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.221023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 829/1654] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:14 2022] DELETE /identity/v3/projects/e24dd4aa9a164db392143ccd6996df8a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:15.225177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2e9c5-cac6-44f8-8c94-c6d9a958e706 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:15.238824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2e9c5-cac6-44f8-8c94-c6d9a958e706 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:15.473451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:15.475245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:15.475679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:15.476085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] PATH_INFO: `/v3/users/6f1695ef7e3d49fe9943b14d88e03ff9` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:15.477919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f1695ef7e3d49fe9943b14d88e03ff9)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:15.479331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 02 03:34:15.479331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jul 02 03:34:15.479331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: Jul 02 03:34:15.481048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:15.558470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:15.657124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f1695ef7e3d49fe9943b14d88e03ff9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.658298 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f1695ef7e3d49fe9943b14d88e03ff9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.675329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f1695ef7e3d49fe9943b14d88e03ff9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.734726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f1695ef7e3d49fe9943b14d88e03ff9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.734726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a89244fd-0e6f-43bb-9a83-33116955c449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f1695ef7e3d49fe9943b14d88e03ff9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:15.775559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 826/1655] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:15 2022] DELETE /identity/v3/users/6f1695ef7e3d49fe9943b14d88e03ff9 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:17.027317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:17.030642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:17.065586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858485cc-052a-4441-8efd-08362893de07 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:17.229998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:17.231611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:17.232556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:17.232556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] PATH_INFO: `/v3/projects/eaa7b24c9e98491da5f783cb50f5a638` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:17.234644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa7b24c9e98491da5f783cb50f5a638)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:17.238049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:17.299985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:17.327764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa7b24c9e98491da5f783cb50f5a638', 'request_id': 'req-e215ec2c-3c91-4445-ae03-2b638523eb14'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:17.351723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa7b24c9e98491da5f783cb50f5a638', 'request_id': 'req-e215ec2c-3c91-4445-ae03-2b638523eb14'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:17.368785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa7b24c9e98491da5f783cb50f5a638', 'request_id': 'req-e215ec2c-3c91-4445-ae03-2b638523eb14'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:17.391073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] The token cache is being invalidate because project eaa7b24c9e98491da5f783cb50f5a638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:17.391398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:17.392361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e215ec2c-3c91-4445-ae03-2b638523eb14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:17.394984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 830/1656] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:17 2022] DELETE /identity/v3/projects/eaa7b24c9e98491da5f783cb50f5a638 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:17.400158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5a6a-7ab0-408c-a2ac-640193289d5c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:17.401055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5a6a-7ab0-408c-a2ac-640193289d5c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:17.401605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5a6a-7ab0-408c-a2ac-640193289d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:17.440529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-609d5a6a-7ab0-408c-a2ac-640193289d5c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:17.441411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-609d5a6a-7ab0-408c-a2ac-640193289d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:17.585727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 827/1657] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:17.593682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2f0b-6616-47d8-84cb-1973e6bb1bd5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:17.593977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2f0b-6616-47d8-84cb-1973e6bb1bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:17.594262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2f0b-6616-47d8-84cb-1973e6bb1bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:17.624686 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-ab0e2f0b-6616-47d8-84cb-1973e6bb1bd5 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:17.626660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ab0e2f0b-6616-47d8-84cb-1973e6bb1bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:17.637337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 831/1658] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:17.641780 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5a6a-7ab0-408c-a2ac-640193289d5c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:17.657113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5a6a-7ab0-408c-a2ac-640193289d5c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:17.667383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebff322-dbb7-4af8-bd50-ff1ba66dc62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:17.681226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebff322-dbb7-4af8-bd50-ff1ba66dc62d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:17.686453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebff322-dbb7-4af8-bd50-ff1ba66dc62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:17.686882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebff322-dbb7-4af8-bd50-ff1ba66dc62d admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:17.694605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebff322-dbb7-4af8-bd50-ff1ba66dc62d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:17.695035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebff322-dbb7-4af8-bd50-ff1ba66dc62d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:17.700137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebff322-dbb7-4af8-bd50-ff1ba66dc62d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:17.757059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 828/1659] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:17.761373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e2f0b-6616-47d8-84cb-1973e6bb1bd5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:17.791075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e2f0b-6616-47d8-84cb-1973e6bb1bd5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:17.822606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483a66a-ed20-41b9-a4b2-01fa89dc3e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:17.823791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483a66a-ed20-41b9-a4b2-01fa89dc3e22 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:17.824076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483a66a-ed20-41b9-a4b2-01fa89dc3e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:17.824360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483a66a-ed20-41b9-a4b2-01fa89dc3e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:17.825091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483a66a-ed20-41b9-a4b2-01fa89dc3e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:17.832774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483a66a-ed20-41b9-a4b2-01fa89dc3e22 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:17.885503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 832/1660] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:17 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:17.888978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebff322-dbb7-4af8-bd50-ff1ba66dc62d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:17.915804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebff322-dbb7-4af8-bd50-ff1ba66dc62d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:18.046633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be15f9-3c34-458c-aafb-69228ad8c51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:18.047370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be15f9-3c34-458c-aafb-69228ad8c51a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.047656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be15f9-3c34-458c-aafb-69228ad8c51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.047931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be15f9-3c34-458c-aafb-69228ad8c51a admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.048642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be15f9-3c34-458c-aafb-69228ad8c51a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:18.050638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be15f9-3c34-458c-aafb-69228ad8c51a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:18.099249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 829/1661] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:17 2022] POST /identity/v3/users => generated 291 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:18.103474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483a66a-ed20-41b9-a4b2-01fa89dc3e22 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:18.118929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483a66a-ed20-41b9-a4b2-01fa89dc3e22 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:18.138391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70013d9-e900-4f61-8256-9d87cc6da510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:18.139120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70013d9-e900-4f61-8256-9d87cc6da510 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.139613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70013d9-e900-4f61-8256-9d87cc6da510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.139896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70013d9-e900-4f61-8256-9d87cc6da510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.140640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70013d9-e900-4f61-8256-9d87cc6da510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:18.144128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70013d9-e900-4f61-8256-9d87cc6da510 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:18.161712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 833/1662] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:18.165761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be15f9-3c34-458c-aafb-69228ad8c51a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:18.180276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be15f9-3c34-458c-aafb-69228ad8c51a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:18.191655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc22ff2-03fe-4aa8-8eef-93a9024f4e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:18.192558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc22ff2-03fe-4aa8-8eef-93a9024f4e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.192973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc22ff2-03fe-4aa8-8eef-93a9024f4e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.193379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc22ff2-03fe-4aa8-8eef-93a9024f4e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.194267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc22ff2-03fe-4aa8-8eef-93a9024f4e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:18.196132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc22ff2-03fe-4aa8-8eef-93a9024f4e54 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:18.209329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 830/1663] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:18.214135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70013d9-e900-4f61-8256-9d87cc6da510 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:18.230984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70013d9-e900-4f61-8256-9d87cc6da510 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:18.243017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df651033-0cb8-4a9b-a4ce-0217f064d5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:18.244035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df651033-0cb8-4a9b-a4ce-0217f064d5a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.246319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df651033-0cb8-4a9b-a4ce-0217f064d5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.246631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df651033-0cb8-4a9b-a4ce-0217f064d5a8 admin admin] PATH_INFO: `/v3/projects/ced4717b9d1d48e0b3371d50b81a3770/users/7485a53bf94b424ca45a9be4d0061e5e/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.279349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df651033-0cb8-4a9b-a4ce-0217f064d5a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ced4717b9d1d48e0b3371d50b81a3770, user_id=7485a53bf94b424ca45a9be4d0061e5e, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:18.282369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df651033-0cb8-4a9b-a4ce-0217f064d5a8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:18.303661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 834/1664] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:18 2022] PUT /identity/v3/projects/ced4717b9d1d48e0b3371d50b81a3770/users/7485a53bf94b424ca45a9be4d0061e5e/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:18.315023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17392b-5f42-4eae-8c60-ff3d03066c82 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.315352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17392b-5f42-4eae-8c60-ff3d03066c82 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.315636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17392b-5f42-4eae-8c60-ff3d03066c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.322317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:18.365676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cc17392b-5f42-4eae-8c60-ff3d03066c82 None None] MFA Rules not processed for user `7485a53bf94b424ca45a9be4d0061e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:18.379058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3c8c2-a743-460d-812e-f3f90634c077 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.379384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3c8c2-a743-460d-812e-f3f90634c077 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.379671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3c8c2-a743-460d-812e-f3f90634c077 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.381513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 835/1665] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:18.384289 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cc17392b-5f42-4eae-8c60-ff3d03066c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:18.463918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2649a2f-43be-4a2c-a6ac-341ab8cdbfda None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.464083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2649a2f-43be-4a2c-a6ac-341ab8cdbfda None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.464311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2649a2f-43be-4a2c-a6ac-341ab8cdbfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.506234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d2649a2f-43be-4a2c-a6ac-341ab8cdbfda None None] MFA Rules not processed for user `6ee7acbd4c5545c1885faffb71f6306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:18.509257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d2649a2f-43be-4a2c-a6ac-341ab8cdbfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:18.538076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 831/1666] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:18.539850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17392b-5f42-4eae-8c60-ff3d03066c82 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:18.628331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17392b-5f42-4eae-8c60-ff3d03066c82 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:18.633739 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154984e3-bf72-4449-a3ed-10941ff0a169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:18.634776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154984e3-bf72-4449-a3ed-10941ff0a169 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.635538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154984e3-bf72-4449-a3ed-10941ff0a169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.637150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154984e3-bf72-4449-a3ed-10941ff0a169 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.638061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154984e3-bf72-4449-a3ed-10941ff0a169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:18.640281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154984e3-bf72-4449-a3ed-10941ff0a169 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:18.679325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 836/1667] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:34:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:18.679969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 832/1668] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:18 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:18.684686 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2649a2f-43be-4a2c-a6ac-341ab8cdbfda None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:18.705078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2649a2f-43be-4a2c-a6ac-341ab8cdbfda None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:18.754008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b533b-6c5a-4fb7-910f-8634b9084d5b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.756042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b533b-6c5a-4fb7-910f-8634b9084d5b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.757454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b533b-6c5a-4fb7-910f-8634b9084d5b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.762727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 833/1669] 10.222.0.42 () {54 vars in 960 bytes} [Sat Jul 2 03:34:18 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:18.777515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f43ea-8b48-4ef9-aab8-66771f2abf87 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.777875 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f43ea-8b48-4ef9-aab8-66771f2abf87 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.778404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f43ea-8b48-4ef9-aab8-66771f2abf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.831318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-771f43ea-8b48-4ef9-aab8-66771f2abf87 None None] MFA Rules not processed for user `6ee7acbd4c5545c1885faffb71f6306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:18.833491 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-771f43ea-8b48-4ef9-aab8-66771f2abf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:18.849356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:18.864107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e52bb-9712-4f35-a985-231f76ba1083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:18.864730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e52bb-9712-4f35-a985-231f76ba1083 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.865129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e52bb-9712-4f35-a985-231f76ba1083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.865281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e52bb-9712-4f35-a985-231f76ba1083 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.866045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e52bb-9712-4f35-a985-231f76ba1083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:18.868582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e52bb-9712-4f35-a985-231f76ba1083 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:18.912706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 837/1670] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:18 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:18.913851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e52bb-9712-4f35-a985-231f76ba1083 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:18.926629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e52bb-9712-4f35-a985-231f76ba1083 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:18.933930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4744250-3fe6-4a79-98da-4417d9d181b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:18.934797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4744250-3fe6-4a79-98da-4417d9d181b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:18.935105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4744250-3fe6-4a79-98da-4417d9d181b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:18.939169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4744250-3fe6-4a79-98da-4417d9d181b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:18.939995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4744250-3fe6-4a79-98da-4417d9d181b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:18.943798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4744250-3fe6-4a79-98da-4417d9d181b6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:18.965521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 838/1671] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:18.966511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4744250-3fe6-4a79-98da-4417d9d181b6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:18.978378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 834/1672] 10.222.0.42 () {58 vars in 1057 bytes} [Sat Jul 2 03:34:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:18.985693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4744250-3fe6-4a79-98da-4417d9d181b6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:18.990180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f43ea-8b48-4ef9-aab8-66771f2abf87 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:19.001701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf93f-8150-4b2f-9dfe-b7f375128824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:19.003061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf93f-8150-4b2f-9dfe-b7f375128824 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.003541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf93f-8150-4b2f-9dfe-b7f375128824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.004855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddf93f-8150-4b2f-9dfe-b7f375128824 admin admin] PATH_INFO: `/v3/projects/9aaaef2d19204bc8909aee5b2b330882/users/98b69773f2b44c39913a34d884a8c362/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.005599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f43ea-8b48-4ef9-aab8-66771f2abf87 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:19.018574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4231c-9cb0-45cc-b875-fdcf7c6f5651 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:19.019518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4231c-9cb0-45cc-b875-fdcf7c6f5651 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.020006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4231c-9cb0-45cc-b875-fdcf7c6f5651 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.020463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4231c-9cb0-45cc-b875-fdcf7c6f5651 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.041740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4231c-9cb0-45cc-b875-fdcf7c6f5651 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:19.043645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4231c-9cb0-45cc-b875-fdcf7c6f5651 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:19.051135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf93f-8150-4b2f-9dfe-b7f375128824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aaaef2d19204bc8909aee5b2b330882, user_id=98b69773f2b44c39913a34d884a8c362, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:19.051453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:19.055388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddf93f-8150-4b2f-9dfe-b7f375128824 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:19.063723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 835/1673] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:19.088317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 839/1674] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:18 2022] PUT /identity/v3/projects/9aaaef2d19204bc8909aee5b2b330882/users/98b69773f2b44c39913a34d884a8c362/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:19.092160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4231c-9cb0-45cc-b875-fdcf7c6f5651 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:19.111275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4231c-9cb0-45cc-b875-fdcf7c6f5651 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:19.237759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a233bcf-818f-45db-a3e0-a30cd3371fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:19.238473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233bcf-818f-45db-a3e0-a30cd3371fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.238757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233bcf-818f-45db-a3e0-a30cd3371fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.239035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233bcf-818f-45db-a3e0-a30cd3371fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.240044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a233bcf-818f-45db-a3e0-a30cd3371fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:19.241843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a233bcf-818f-45db-a3e0-a30cd3371fe6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:19.260144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 836/1675] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:34:19.261083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf93f-8150-4b2f-9dfe-b7f375128824 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:19.276661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddf93f-8150-4b2f-9dfe-b7f375128824 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:19.284154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78657c6f-7744-4ea1-902b-e2c846383223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:19.286116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78657c6f-7744-4ea1-902b-e2c846383223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.286933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78657c6f-7744-4ea1-902b-e2c846383223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.287345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78657c6f-7744-4ea1-902b-e2c846383223 admin admin] PATH_INFO: `/v3/projects/9aaaef2d19204bc8909aee5b2b330882/users/98b69773f2b44c39913a34d884a8c362/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.290943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78657c6f-7744-4ea1-902b-e2c846383223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aaaef2d19204bc8909aee5b2b330882, user_id=98b69773f2b44c39913a34d884a8c362, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:19.295864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78657c6f-7744-4ea1-902b-e2c846383223 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:19.317224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 840/1676] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:19 2022] PUT /identity/v3/projects/9aaaef2d19204bc8909aee5b2b330882/users/98b69773f2b44c39913a34d884a8c362/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:19.328232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05153f-cba7-4c28-bb88-bc0ec9940e37 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.328760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05153f-cba7-4c28-bb88-bc0ec9940e37 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.329129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05153f-cba7-4c28-bb88-bc0ec9940e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.361786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ed05153f-cba7-4c28-bb88-bc0ec9940e37 None None] MFA Rules not processed for user `98b69773f2b44c39913a34d884a8c362`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:19.363337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ed05153f-cba7-4c28-bb88-bc0ec9940e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:19.494464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 837/1677] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:19.503596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78657c6f-7744-4ea1-902b-e2c846383223 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:19.508178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-78657c6f-7744-4ea1-902b-e2c846383223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:19.537430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133234a-a363-4d6b-8008-a5492cdfb676 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.537733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133234a-a363-4d6b-8008-a5492cdfb676 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.537986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133234a-a363-4d6b-8008-a5492cdfb676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.584094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f133234a-a363-4d6b-8008-a5492cdfb676 None None] MFA Rules not processed for user `c941dc00e0584666aba13263b8c1fcb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:19.585678 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f133234a-a363-4d6b-8008-a5492cdfb676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:19.589301 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78657c6f-7744-4ea1-902b-e2c846383223 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:19.593598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438da685-3d1e-459e-b28e-3ed8571231d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:19.594486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438da685-3d1e-459e-b28e-3ed8571231d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.594770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438da685-3d1e-459e-b28e-3ed8571231d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.595049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438da685-3d1e-459e-b28e-3ed8571231d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.608089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438da685-3d1e-459e-b28e-3ed8571231d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:19.609624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438da685-3d1e-459e-b28e-3ed8571231d4 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:19.622950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 841/1678] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:19.709723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 838/1679] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:19.724115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bb2d0-74d7-48b1-aeda-acce6b304941 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.727883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bb2d0-74d7-48b1-aeda-acce6b304941 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.729509 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bb2d0-74d7-48b1-aeda-acce6b304941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.731416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133234a-a363-4d6b-8008-a5492cdfb676 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:19.755794 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2b7bb2d0-74d7-48b1-aeda-acce6b304941 None None] MFA Rules not processed for user `c941dc00e0584666aba13263b8c1fcb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:19.757256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133234a-a363-4d6b-8008-a5492cdfb676 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:19.757374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2b7bb2d0-74d7-48b1-aeda-acce6b304941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:19.766004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee5463-896e-42ba-ab01-9259fad385f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:19.769184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee5463-896e-42ba-ab01-9259fad385f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.769994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee5463-896e-42ba-ab01-9259fad385f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.771024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee5463-896e-42ba-ab01-9259fad385f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.775755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 842/1680] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:19.798848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:19.828051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b66b1-e4ec-4e73-a9e9-481f8c73cd0b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.828778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b66b1-e4ec-4e73-a9e9-481f8c73cd0b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.829176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b66b1-e4ec-4e73-a9e9-481f8c73cd0b None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.831201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 843/1681] 10.222.0.42 () {54 vars in 961 bytes} [Sat Jul 2 03:34:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:19.853738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d842e-cfcd-4000-b570-507a4a7ef3b1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:19.854266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d842e-cfcd-4000-b570-507a4a7ef3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:19.854646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d842e-cfcd-4000-b570-507a4a7ef3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:19.895895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a95d842e-cfcd-4000-b570-507a4a7ef3b1 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:19.899862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a95d842e-cfcd-4000-b570-507a4a7ef3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:19.923709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee5463-896e-42ba-ab01-9259fad385f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:19.925504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee5463-896e-42ba-ab01-9259fad385f3 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:19.960758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 839/1682] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:20.028155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 844/1683] 10.222.0.42 () {58 vars in 1058 bytes} [Sat Jul 2 03:34:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:20.041430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee5463-896e-42ba-ab01-9259fad385f3 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:20.080214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee5463-896e-42ba-ab01-9259fad385f3 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:20.102467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca070fc-2ff3-464a-84cb-f7d72fe479b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:20.103959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca070fc-2ff3-464a-84cb-f7d72fe479b6 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:20.104408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca070fc-2ff3-464a-84cb-f7d72fe479b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:20.104843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca070fc-2ff3-464a-84cb-f7d72fe479b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:20.144553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca070fc-2ff3-464a-84cb-f7d72fe479b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:20.146280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca070fc-2ff3-464a-84cb-f7d72fe479b6 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:20.172687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 840/1684] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:20.749761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45a2e-eefe-402a-8323-8c7c12ba6441 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:20.750276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45a2e-eefe-402a-8323-8c7c12ba6441 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:20.750639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d45a2e-eefe-402a-8323-8c7c12ba6441 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:20.752978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 845/1685] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:34:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:20.763613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc17fc-2241-46e7-8593-4f9d72ecbdfb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:20.764064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc17fc-2241-46e7-8593-4f9d72ecbdfb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:20.765434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc17fc-2241-46e7-8593-4f9d72ecbdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:20.799691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fcfc17fc-2241-46e7-8593-4f9d72ecbdfb None None] MFA Rules not processed for user `6ee7acbd4c5545c1885faffb71f6306b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:20.801165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fcfc17fc-2241-46e7-8593-4f9d72ecbdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:20.837682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d45a2e-eefe-402a-8323-8c7c12ba6441 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:20.839419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e3d45a2e-eefe-402a-8323-8c7c12ba6441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:20.917430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:21.001920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 841/1686] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:34:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:21.003629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e2074-1777-4bdf-a402-c75d741c2e4e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:21.003931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e2074-1777-4bdf-a402-c75d741c2e4e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:21.004215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e2074-1777-4bdf-a402-c75d741c2e4e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:21.006954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 842/1687] 10.222.0.42 () {54 vars in 912 bytes} [Sat Jul 2 03:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:21.014309 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62cfd0-92f7-46b6-a96f-c283a3ad6f1b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:21.014669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62cfd0-92f7-46b6-a96f-c283a3ad6f1b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:21.014876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62cfd0-92f7-46b6-a96f-c283a3ad6f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:21.037461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d45a2e-eefe-402a-8323-8c7c12ba6441 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:21.051550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:21.052544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:21.052930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:21.053222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] PATH_INFO: `/v3/users/84ba1f7871004be782bc51daf1235c41` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:21.054653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ba1f7871004be782bc51daf1235c41)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:21.056770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:21.063165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0d62cfd0-92f7-46b6-a96f-c283a3ad6f1b None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:21.066874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0d62cfd0-92f7-46b6-a96f-c283a3ad6f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:21.095681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 843/1688] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:34:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:21.096564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62cfd0-92f7-46b6-a96f-c283a3ad6f1b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:21.127104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62cfd0-92f7-46b6-a96f-c283a3ad6f1b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:21.144857 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ea539-3c41-42f5-b84a-7f5311891439 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:21.146486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ea539-3c41-42f5-b84a-7f5311891439 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:21.147796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ea539-3c41-42f5-b84a-7f5311891439 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:21.149782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8ea539-3c41-42f5-b84a-7f5311891439 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:21.167555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ba1f7871004be782bc51daf1235c41'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.169023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ba1f7871004be782bc51daf1235c41'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.174946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ea539-3c41-42f5-b84a-7f5311891439 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:21.180927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8ea539-3c41-42f5-b84a-7f5311891439 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:21.181282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ba1f7871004be782bc51daf1235c41'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.182241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ba1f7871004be782bc51daf1235c41'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.198684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 844/1689] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:21.200502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ea539-3c41-42f5-b84a-7f5311891439 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:21.202584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2d8ea539-3c41-42f5-b84a-7f5311891439 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:21.217009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ba1f7871004be782bc51daf1235c41'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.279578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 846/1690] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:20 2022] DELETE /identity/v3/users/84ba1f7871004be782bc51daf1235c41 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:21.392031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8ea539-3c41-42f5-b84a-7f5311891439 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:21.447485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:21.483909 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb39a0a-7343-47f0-b783-04df3f47cd67 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:21.484626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb39a0a-7343-47f0-b783-04df3f47cd67 service glance] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:21.484811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb39a0a-7343-47f0-b783-04df3f47cd67 service glance] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:21.485624 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb39a0a-7343-47f0-b783-04df3f47cd67 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:21.488118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cfb39a0a-7343-47f0-b783-04df3f47cd67 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:21.606159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb39a0a-7343-47f0-b783-04df3f47cd67 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:21.607709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb39a0a-7343-47f0-b783-04df3f47cd67 service glance] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:21.627634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 845/1691] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:21.662935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:21.664533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:21.778521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aea91c-cafe-4b4b-984b-e285af42f53a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:21.785568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:21.787559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:21.788066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:21.788950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] PATH_INFO: `/v3/projects/6aae4119457442c2849db1540801d3d8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:21.791266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aae4119457442c2849db1540801d3d8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:21.794896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:21.866741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:21.873766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aae4119457442c2849db1540801d3d8', 'request_id': 'req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.897587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aae4119457442c2849db1540801d3d8', 'request_id': 'req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.909584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aae4119457442c2849db1540801d3d8', 'request_id': 'req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.948049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] The token cache is being invalidate because project 6aae4119457442c2849db1540801d3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:21.948049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.959226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-12515d8c-8bce-4ad5-b2d0-9e3871a63eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:21.963163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 847/1692] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:21 2022] DELETE /identity/v3/projects/6aae4119457442c2849db1540801d3d8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:21.966834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb39a0a-7343-47f0-b783-04df3f47cd67 service glance] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:21.979728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb39a0a-7343-47f0-b783-04df3f47cd67 service glance] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:22.097770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:22.099127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:22.099685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:22.100170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] PATH_INFO: `/v3/users/4503bfcd35aa4936902deba9f2228894` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:22.103174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4503bfcd35aa4936902deba9f2228894)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:22.106189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:22.205735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4503bfcd35aa4936902deba9f2228894'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:22.206684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4503bfcd35aa4936902deba9f2228894'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:22.226475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4503bfcd35aa4936902deba9f2228894'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:22.247146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4503bfcd35aa4936902deba9f2228894'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:22.248051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f281b0e8-3f81-4ea2-ab27-dfe8380d8d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4503bfcd35aa4936902deba9f2228894'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:22.254172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:22.268266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 846/1693] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:21 2022] DELETE /identity/v3/users/4503bfcd35aa4936902deba9f2228894 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:22.416375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2c79a-e51f-49b9-9990-454d42d27dbe None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:22.416908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2c79a-e51f-49b9-9990-454d42d27dbe None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:22.417474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2c79a-e51f-49b9-9990-454d42d27dbe None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:22.419678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 848/1694] 10.222.0.42 () {54 vars in 912 bytes} [Sat Jul 2 03:34:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:22.427535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dd49a-297c-4338-b6a9-2e2e1ed12e2d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:22.427844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dd49a-297c-4338-b6a9-2e2e1ed12e2d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:22.428103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dd49a-297c-4338-b6a9-2e2e1ed12e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:22.544793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-310dd49a-297c-4338-b6a9-2e2e1ed12e2d None None] MFA Rules not processed for user `d67be26446324c91b956e9d930e04b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:22.546436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-310dd49a-297c-4338-b6a9-2e2e1ed12e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:22.743370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2c79a-e51f-49b9-9990-454d42d27dbe None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:22.746930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-40d2c79a-e51f-49b9-9990-454d42d27dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:22.766614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 847/1695] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:34:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:22.814192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dd49a-297c-4338-b6a9-2e2e1ed12e2d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:22.816666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-310dd49a-297c-4338-b6a9-2e2e1ed12e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:22.838500 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dd49a-297c-4338-b6a9-2e2e1ed12e2d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:22.859316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe5be7-8d1d-4808-892a-608673558cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:22.862468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe5be7-8d1d-4808-892a-608673558cdd service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:22.865750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe5be7-8d1d-4808-892a-608673558cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:22.865999 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe5be7-8d1d-4808-892a-608673558cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:22.897754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe5be7-8d1d-4808-892a-608673558cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:22.904900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe5be7-8d1d-4808-892a-608673558cdd service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:22.930009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 848/1696] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:22.982202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2c79a-e51f-49b9-9990-454d42d27dbe None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:23.000001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:23.000957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:23.001245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:23.001520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] PATH_INFO: `/v3/projects/17c03a1303434dc48a16d1a5daa886d1` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:23.004094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c03a1303434dc48a16d1a5daa886d1)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:23.006111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:23.133194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c03a1303434dc48a16d1a5daa886d1', 'request_id': 'req-d6e825ad-4b22-431b-831a-6becdff68f8d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:23.153047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c03a1303434dc48a16d1a5daa886d1', 'request_id': 'req-d6e825ad-4b22-431b-831a-6becdff68f8d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:23.163948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c03a1303434dc48a16d1a5daa886d1', 'request_id': 'req-d6e825ad-4b22-431b-831a-6becdff68f8d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:23.218635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] The token cache is being invalidate because project 17c03a1303434dc48a16d1a5daa886d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:23.218936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:23.219725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d6e825ad-4b22-431b-831a-6becdff68f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:23.223727 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 849/1697] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:22 2022] DELETE /identity/v3/projects/17c03a1303434dc48a16d1a5daa886d1 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:23.228768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d52b-bf0a-46c8-9e6a-8ae94d453928 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:23.229269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d52b-bf0a-46c8-9e6a-8ae94d453928 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:23.229425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d52b-bf0a-46c8-9e6a-8ae94d453928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:23.313695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a5f5d52b-bf0a-46c8-9e6a-8ae94d453928 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:23.322903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a5f5d52b-bf0a-46c8-9e6a-8ae94d453928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:23.580470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646562e6-a65b-43bd-8565-19822c827e7f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:23.580762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646562e6-a65b-43bd-8565-19822c827e7f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:23.581107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646562e6-a65b-43bd-8565-19822c827e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:23.581797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 849/1698] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:23.620531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-646562e6-a65b-43bd-8565-19822c827e7f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:23.621791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-646562e6-a65b-43bd-8565-19822c827e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:23.651401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 850/1699] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:23.656603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5d52b-bf0a-46c8-9e6a-8ae94d453928 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:23.672036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5d52b-bf0a-46c8-9e6a-8ae94d453928 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:23.688053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde58d8-a6b9-4311-950f-36c131831f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:23.689110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde58d8-a6b9-4311-950f-36c131831f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:23.689110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde58d8-a6b9-4311-950f-36c131831f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:23.689339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde58d8-a6b9-4311-950f-36c131831f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:23.690264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde58d8-a6b9-4311-950f-36c131831f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:23.716807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde58d8-a6b9-4311-950f-36c131831f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:23.716807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde58d8-a6b9-4311-950f-36c131831f84 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:23.718440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8e44e-c8ec-410d-985f-7bb1e5e1ed26 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:23.718734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8e44e-c8ec-410d-985f-7bb1e5e1ed26 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:23.719020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8e44e-c8ec-410d-985f-7bb1e5e1ed26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:23.780156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 850/1700] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:23.781171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde58d8-a6b9-4311-950f-36c131831f84 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:23.788802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a8d8e44e-c8ec-410d-985f-7bb1e5e1ed26 None None] MFA Rules not processed for user `9028859afbab40df9fc4e5ea6d876931`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:23.790836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a8d8e44e-c8ec-410d-985f-7bb1e5e1ed26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:23.823692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde58d8-a6b9-4311-950f-36c131831f84 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:23.837584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680db88a-fce1-42b6-b133-c745ffca7147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:23.840353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680db88a-fce1-42b6-b133-c745ffca7147 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:23.840912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680db88a-fce1-42b6-b133-c745ffca7147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:23.841619 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680db88a-fce1-42b6-b133-c745ffca7147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:23.843174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680db88a-fce1-42b6-b133-c745ffca7147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:23.863437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680db88a-fce1-42b6-b133-c745ffca7147 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:23.937425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 851/1701] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:23 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:23.939140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680db88a-fce1-42b6-b133-c745ffca7147 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:23.939833 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:23.998337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 851/1702] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 02 03:34:24.009745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f8f9e-232d-4569-abe4-8677d207895f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:24.010040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f8f9e-232d-4569-abe4-8677d207895f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:24.015831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f8f9e-232d-4569-abe4-8677d207895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:24.059022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-638f8f9e-232d-4569-abe4-8677d207895f None None] MFA Rules not processed for user `9028859afbab40df9fc4e5ea6d876931`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:24.060317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-638f8f9e-232d-4569-abe4-8677d207895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:24.063861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:24.116712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680db88a-fce1-42b6-b133-c745ffca7147 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:24.136824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf57cc-c7fb-430f-8b40-83aea17859fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:24.137692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf57cc-c7fb-430f-8b40-83aea17859fc admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:24.138073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf57cc-c7fb-430f-8b40-83aea17859fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:24.138560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf57cc-c7fb-430f-8b40-83aea17859fc admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:24.140633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf57cc-c7fb-430f-8b40-83aea17859fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:24.144852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf57cc-c7fb-430f-8b40-83aea17859fc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:24.195810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 852/1703] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:23 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:24.200475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf57cc-c7fb-430f-8b40-83aea17859fc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:24.240139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 852/1704] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:24.240863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf57cc-c7fb-430f-8b40-83aea17859fc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:24.271005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55c7b-c87b-400d-ba99-abfae565009d None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:24.271196 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55c7b-c87b-400d-ba99-abfae565009d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:24.272024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55c7b-c87b-400d-ba99-abfae565009d None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:24.274117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 853/1705] 10.222.0.42 () {54 vars in 961 bytes} [Sat Jul 2 03:34:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 02 03:34:24.288593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7658e9d-bc6e-464b-a1e4-dbbea45a820c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:24.289597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7658e9d-bc6e-464b-a1e4-dbbea45a820c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:24.289888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7658e9d-bc6e-464b-a1e4-dbbea45a820c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:24.290815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7658e9d-bc6e-464b-a1e4-dbbea45a820c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:24.292678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648baf6-aea6-45b5-862d-f451d5501df8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:24.293040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648baf6-aea6-45b5-862d-f451d5501df8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:24.293373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648baf6-aea6-45b5-862d-f451d5501df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:24.306846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7658e9d-bc6e-464b-a1e4-dbbea45a820c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:24.317704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7658e9d-bc6e-464b-a1e4-dbbea45a820c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:24.368572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0648baf6-aea6-45b5-862d-f451d5501df8 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:24.369865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0648baf6-aea6-45b5-862d-f451d5501df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:24.378628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 853/1706] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:24.379670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7658e9d-bc6e-464b-a1e4-dbbea45a820c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:24.418355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7658e9d-bc6e-464b-a1e4-dbbea45a820c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:24.466780 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fc329-a290-44b8-9ff9-bc929748aa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:24.467529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fc329-a290-44b8-9ff9-bc929748aa79 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:24.467808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fc329-a290-44b8-9ff9-bc929748aa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:24.468092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fc329-a290-44b8-9ff9-bc929748aa79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:24.468877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fc329-a290-44b8-9ff9-bc929748aa79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:24.478112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fc329-a290-44b8-9ff9-bc929748aa79 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:24.556418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 854/1707] 10.222.0.42 () {58 vars in 1058 bytes} [Sat Jul 2 03:34:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:24.557441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 854/1708] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:24.561832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648baf6-aea6-45b5-862d-f451d5501df8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:24.586030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648baf6-aea6-45b5-862d-f451d5501df8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:24.587419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fc329-a290-44b8-9ff9-bc929748aa79 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:24.596442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8d328-3720-45af-ad91-09bc36b3835e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:24.598512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8d328-3720-45af-ad91-09bc36b3835e admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:24.598823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8d328-3720-45af-ad91-09bc36b3835e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:24.600453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8d328-3720-45af-ad91-09bc36b3835e admin admin] PATH_INFO: `/v3/projects/c9fd9b23df55413aa29f121b9d22df1e/users/eac4e0a79f614d3791e5df9b8af35204/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:24.641557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8d328-3720-45af-ad91-09bc36b3835e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9fd9b23df55413aa29f121b9d22df1e, user_id=eac4e0a79f614d3791e5df9b8af35204, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:24.649322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fc329-a290-44b8-9ff9-bc929748aa79 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:24.653121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8d328-3720-45af-ad91-09bc36b3835e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:24.681851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0665ee-0df5-4e6b-8b17-cd5ff8497511 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:24.684453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0665ee-0df5-4e6b-8b17-cd5ff8497511 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:24.687890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0665ee-0df5-4e6b-8b17-cd5ff8497511 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:24.690301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0665ee-0df5-4e6b-8b17-cd5ff8497511 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:24.690686 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 855/1709] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:24 2022] PUT /identity/v3/projects/c9fd9b23df55413aa29f121b9d22df1e/users/eac4e0a79f614d3791e5df9b8af35204/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:24.702667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edefa9-7cb8-436b-a52e-c149c67980b8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:24.705191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edefa9-7cb8-436b-a52e-c149c67980b8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:24.705940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edefa9-7cb8-436b-a52e-c149c67980b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:24.733407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0665ee-0df5-4e6b-8b17-cd5ff8497511 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:24.735394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0665ee-0df5-4e6b-8b17-cd5ff8497511 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:24.771348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 855/1710] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:24.804755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c9edefa9-7cb8-436b-a52e-c149c67980b8 None None] MFA Rules not processed for user `eac4e0a79f614d3791e5df9b8af35204`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:24.808805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c9edefa9-7cb8-436b-a52e-c149c67980b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:24.927152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:24.967795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 856/1711] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:24.971957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0665ee-0df5-4e6b-8b17-cd5ff8497511 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:24.990002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0665ee-0df5-4e6b-8b17-cd5ff8497511 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:25.223760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35e21f-1f4e-41e2-8ec6-194563a8fb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:25.224469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35e21f-1f4e-41e2-8ec6-194563a8fb95 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:25.224754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35e21f-1f4e-41e2-8ec6-194563a8fb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:25.225028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35e21f-1f4e-41e2-8ec6-194563a8fb95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:25.226118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35e21f-1f4e-41e2-8ec6-194563a8fb95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:25.228689 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35e21f-1f4e-41e2-8ec6-194563a8fb95 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:25.315754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edefa9-7cb8-436b-a52e-c149c67980b8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:25.317711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c9edefa9-7cb8-436b-a52e-c149c67980b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:25.326618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 856/1712] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:24 2022] POST /identity/v3/projects => generated 371 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:25.329782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35e21f-1f4e-41e2-8ec6-194563a8fb95 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:25.518547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edefa9-7cb8-436b-a52e-c149c67980b8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:25.570001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35e21f-1f4e-41e2-8ec6-194563a8fb95 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:25.581709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343f4f3-070a-4760-ad5b-97599b4e2f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:25.582514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343f4f3-070a-4760-ad5b-97599b4e2f61 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:25.582895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343f4f3-070a-4760-ad5b-97599b4e2f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:25.583255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343f4f3-070a-4760-ad5b-97599b4e2f61 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:25.584079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343f4f3-070a-4760-ad5b-97599b4e2f61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:25.589311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343f4f3-070a-4760-ad5b-97599b4e2f61 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:25.638237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 857/1713] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:25 2022] POST /identity/v3/users => generated 292 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:25.638237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1c30d-f339-4465-8fcb-fcadf0c4fa7c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:25.638237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1c30d-f339-4465-8fcb-fcadf0c4fa7c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:25.638237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1c30d-f339-4465-8fcb-fcadf0c4fa7c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:25.656535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 858/1714] 10.222.0.42 () {54 vars in 912 bytes} [Sat Jul 2 03:34:25 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:25.657617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1c30d-f339-4465-8fcb-fcadf0c4fa7c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:25.692636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1c30d-f339-4465-8fcb-fcadf0c4fa7c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:25.708930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b949c-edfc-497d-9ee0-bc831e490773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:25.709630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b949c-edfc-497d-9ee0-bc831e490773 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:25.709923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b949c-edfc-497d-9ee0-bc831e490773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:25.710197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b949c-edfc-497d-9ee0-bc831e490773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:25.711009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b949c-edfc-497d-9ee0-bc831e490773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:25.712756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b949c-edfc-497d-9ee0-bc831e490773 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:25.727668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:25.735070 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b1f7d-7e4e-42a7-a14c-e0b8e3adafed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:25.735789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b1f7d-7e4e-42a7-a14c-e0b8e3adafed service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:25.736075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b1f7d-7e4e-42a7-a14c-e0b8e3adafed service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:25.736347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b1f7d-7e4e-42a7-a14c-e0b8e3adafed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:25.738661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-679b1f7d-7e4e-42a7-a14c-e0b8e3adafed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:25.757012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 859/1715] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:25.758594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00675-d198-4e3e-bd3c-7e4d6ff2dc22 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:25.758880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00675-d198-4e3e-bd3c-7e4d6ff2dc22 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:25.760181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda00675-d198-4e3e-bd3c-7e4d6ff2dc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:25.772088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b1f7d-7e4e-42a7-a14c-e0b8e3adafed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:25.774170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b1f7d-7e4e-42a7-a14c-e0b8e3adafed service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:25.812118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fda00675-d198-4e3e-bd3c-7e4d6ff2dc22 None None] MFA Rules not processed for user `c941dc00e0584666aba13263b8c1fcb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:25.813793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fda00675-d198-4e3e-bd3c-7e4d6ff2dc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:25.826652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 857/1716] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:25.828222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a43f0d-377f-47da-aa14-2a6723d26489 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:25.830150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a43f0d-377f-47da-aa14-2a6723d26489 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:25.830857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a43f0d-377f-47da-aa14-2a6723d26489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:25.895741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-10a43f0d-377f-47da-aa14-2a6723d26489 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:25.900085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-10a43f0d-377f-47da-aa14-2a6723d26489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:26.046188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 860/1717] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:26.046188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda00675-d198-4e3e-bd3c-7e4d6ff2dc22 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:26.122128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda00675-d198-4e3e-bd3c-7e4d6ff2dc22 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:26.147803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aab66-721d-4e49-ad44-e02555e8ec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:26.148982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aab66-721d-4e49-ad44-e02555e8ec4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:26.149171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aab66-721d-4e49-ad44-e02555e8ec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:26.149455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4aab66-721d-4e49-ad44-e02555e8ec4a admin admin] PATH_INFO: `/v3/projects/305901ca0f594b4581ab34bb111eef69/users/352a3d88037540688f4dab794bddc356/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:26.216395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4aab66-721d-4e49-ad44-e02555e8ec4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=305901ca0f594b4581ab34bb111eef69, user_id=352a3d88037540688f4dab794bddc356, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:26.229946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4aab66-721d-4e49-ad44-e02555e8ec4a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:26.272630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 861/1718] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:26 2022] PUT /identity/v3/projects/305901ca0f594b4581ab34bb111eef69/users/352a3d88037540688f4dab794bddc356/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:26.276563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aab66-721d-4e49-ad44-e02555e8ec4a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:26.289987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 858/1719] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:34:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:26.306532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a43f0d-377f-47da-aa14-2a6723d26489 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:26.308337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-10a43f0d-377f-47da-aa14-2a6723d26489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:26.346609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:26.493611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a43f0d-377f-47da-aa14-2a6723d26489 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:26.536922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4aab66-721d-4e49-ad44-e02555e8ec4a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:26.552841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a9f0d-c74f-4885-8675-0ddad76be13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:26.554967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a9f0d-c74f-4885-8675-0ddad76be13a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:26.555269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a9f0d-c74f-4885-8675-0ddad76be13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:26.557806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a9f0d-c74f-4885-8675-0ddad76be13a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:26.561066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a9f0d-c74f-4885-8675-0ddad76be13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:26.569394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a9f0d-c74f-4885-8675-0ddad76be13a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:26.626212 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 862/1720] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:26.628564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a9f0d-c74f-4885-8675-0ddad76be13a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:26.694592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d80ce6-bcfa-48b2-a7a1-50b000f4c8c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:26.695675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80ce6-bcfa-48b2-a7a1-50b000f4c8c1 service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:26.695675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80ce6-bcfa-48b2-a7a1-50b000f4c8c1 service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:26.695967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80ce6-bcfa-48b2-a7a1-50b000f4c8c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:26.698433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a9f0d-c74f-4885-8675-0ddad76be13a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:26.712081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21f64f-c784-4741-853a-57cc75c6ff13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:26.714390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21f64f-c784-4741-853a-57cc75c6ff13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:26.714688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21f64f-c784-4741-853a-57cc75c6ff13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:26.714972 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21f64f-c784-4741-853a-57cc75c6ff13 admin admin] PATH_INFO: `/v3/projects/305901ca0f594b4581ab34bb111eef69/users/352a3d88037540688f4dab794bddc356/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:26.726413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21f64f-c784-4741-853a-57cc75c6ff13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305901ca0f594b4581ab34bb111eef69, user_id=352a3d88037540688f4dab794bddc356, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:26.734069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21f64f-c784-4741-853a-57cc75c6ff13 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:26.779392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 863/1721] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:26 2022] PUT /identity/v3/projects/305901ca0f594b4581ab34bb111eef69/users/352a3d88037540688f4dab794bddc356/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:26.791404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111002f0-bf92-4c74-9ce5-46c03f56276b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:26.791703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111002f0-bf92-4c74-9ce5-46c03f56276b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:26.792030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111002f0-bf92-4c74-9ce5-46c03f56276b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:26.857634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-111002f0-bf92-4c74-9ce5-46c03f56276b None None] MFA Rules not processed for user `352a3d88037540688f4dab794bddc356`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:26.865438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-111002f0-bf92-4c74-9ce5-46c03f56276b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:27.027861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d80ce6-bcfa-48b2-a7a1-50b000f4c8c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:27.029467 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d80ce6-bcfa-48b2-a7a1-50b000f4c8c1 service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:27.083873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 859/1722] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 777 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:27.116613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:27.205769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 864/1723] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:27.207118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d80ce6-bcfa-48b2-a7a1-50b000f4c8c1 service glance] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:27.208689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-52d80ce6-bcfa-48b2-a7a1-50b000f4c8c1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:27.248596 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b075cc-4175-4a8c-bc0e-cd60f674e993 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:27.249094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b075cc-4175-4a8c-bc0e-cd60f674e993 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:27.249570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b075cc-4175-4a8c-bc0e-cd60f674e993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:27.293680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-22b075cc-4175-4a8c-bc0e-cd60f674e993 None None] MFA Rules not processed for user `868e2f0e822b430981369d414eee8e53`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:27.317080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-22b075cc-4175-4a8c-bc0e-cd60f674e993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:27.384571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d80ce6-bcfa-48b2-a7a1-50b000f4c8c1 service glance] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:27.450624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:27.529453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 865/1724] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:27.555464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f88ce-c8ca-4be6-9bf5-64fb4ce16513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:27.556714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f88ce-c8ca-4be6-9bf5-64fb4ce16513 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:27.557013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f88ce-c8ca-4be6-9bf5-64fb4ce16513 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:27.560284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f88ce-c8ca-4be6-9bf5-64fb4ce16513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:27.583663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f88ce-c8ca-4be6-9bf5-64fb4ce16513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:27.584802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f88ce-c8ca-4be6-9bf5-64fb4ce16513 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:27.603594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 860/1725] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:27.706361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b943c-c93b-4280-9d5f-73be98124e0b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:27.706837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b943c-c93b-4280-9d5f-73be98124e0b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:27.707045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b943c-c93b-4280-9d5f-73be98124e0b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:27.709381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 866/1726] 10.222.0.42 () {54 vars in 912 bytes} [Sat Jul 2 03:34:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:27.715469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ae368-962d-4819-8989-4d0081d79f19 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:27.715760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ae368-962d-4819-8989-4d0081d79f19 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:27.716020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ae368-962d-4819-8989-4d0081d79f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:27.746942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d18ae368-962d-4819-8989-4d0081d79f19 None None] MFA Rules not processed for user `d67be26446324c91b956e9d930e04b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:27.754924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d18ae368-962d-4819-8989-4d0081d79f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:27.935721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 861/1727] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:34:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:27.970611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b943c-c93b-4280-9d5f-73be98124e0b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:27.973391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-882b943c-c93b-4280-9d5f-73be98124e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:28.000400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b943c-c93b-4280-9d5f-73be98124e0b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:28.010133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef3819-cac8-4ec0-a9c8-72ead09c98a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:28.010792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef3819-cac8-4ec0-a9c8-72ead09c98a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:28.011073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef3819-cac8-4ec0-a9c8-72ead09c98a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:28.011400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef3819-cac8-4ec0-a9c8-72ead09c98a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:28.046099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef3819-cac8-4ec0-a9c8-72ead09c98a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:28.047803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef3819-cac8-4ec0-a9c8-72ead09c98a9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:28.070069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 867/1728] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:28.284418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ae368-962d-4819-8989-4d0081d79f19 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:28.286122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d18ae368-962d-4819-8989-4d0081d79f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:28.449284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef3819-cac8-4ec0-a9c8-72ead09c98a9 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:28.490951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ae368-962d-4819-8989-4d0081d79f19 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:28.500608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62ca3c-4bc1-45a0-99f3-e9be11ded46a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:28.501624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62ca3c-4bc1-45a0-99f3-e9be11ded46a service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:28.502012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62ca3c-4bc1-45a0-99f3-e9be11ded46a service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:28.502389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62ca3c-4bc1-45a0-99f3-e9be11ded46a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:28.517450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62ca3c-4bc1-45a0-99f3-e9be11ded46a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:28.519560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62ca3c-4bc1-45a0-99f3-e9be11ded46a service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:28.570246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 862/1729] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:28.599430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef3819-cac8-4ec0-a9c8-72ead09c98a9 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:28.608063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:28.609357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:28.611977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:28.613791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] PATH_INFO: `/v3/users/eac4e0a79f614d3791e5df9b8af35204` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:28.616556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac4e0a79f614d3791e5df9b8af35204)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:28.622206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:28.747119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac4e0a79f614d3791e5df9b8af35204'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:28.747856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac4e0a79f614d3791e5df9b8af35204'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:28.763733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac4e0a79f614d3791e5df9b8af35204'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:28.782049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:28.798274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac4e0a79f614d3791e5df9b8af35204'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:28.799030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-089cea6b-f0c6-44ad-bcf3-1717775ae8e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac4e0a79f614d3791e5df9b8af35204'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:28.825440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 868/1730] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:28 2022] DELETE /identity/v3/users/eac4e0a79f614d3791e5df9b8af35204 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:28.831152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539dde4-76a5-4f1c-9572-7ae965689025 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:28.832348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539dde4-76a5-4f1c-9572-7ae965689025 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:28.833661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539dde4-76a5-4f1c-9572-7ae965689025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:28.924685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b6fa3-e4df-47da-a12d-2472169b8f2f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:28.925100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b6fa3-e4df-47da-a12d-2472169b8f2f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:28.925569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b6fa3-e4df-47da-a12d-2472169b8f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:28.934754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e539dde4-76a5-4f1c-9572-7ae965689025 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:28.938409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e539dde4-76a5-4f1c-9572-7ae965689025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:28.992057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8f5b6fa3-e4df-47da-a12d-2472169b8f2f None None] MFA Rules not processed for user `112d731047a64efc8e9143c4d471a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:28.993882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8f5b6fa3-e4df-47da-a12d-2472169b8f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:29.130539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 863/1731] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:29.157634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539dde4-76a5-4f1c-9572-7ae965689025 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:29.159593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e539dde4-76a5-4f1c-9572-7ae965689025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:29.184399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:29.238135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 869/1732] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:29.251840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b6fa3-e4df-47da-a12d-2472169b8f2f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:29.253456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8f5b6fa3-e4df-47da-a12d-2472169b8f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:29.372370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539dde4-76a5-4f1c-9572-7ae965689025 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:29.386147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547be73-78ba-43e5-87d2-b7906ec18a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:29.386917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547be73-78ba-43e5-87d2-b7906ec18a56 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:29.387204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547be73-78ba-43e5-87d2-b7906ec18a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:29.387615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547be73-78ba-43e5-87d2-b7906ec18a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:29.415017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547be73-78ba-43e5-87d2-b7906ec18a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:29.429324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b6fa3-e4df-47da-a12d-2472169b8f2f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:29.434115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547be73-78ba-43e5-87d2-b7906ec18a56 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:29.442721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dea3d-758c-42e2-b2e9-eb4a8e35b3aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:29.443537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dea3d-758c-42e2-b2e9-eb4a8e35b3aa service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:29.443830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dea3d-758c-42e2-b2e9-eb4a8e35b3aa service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:29.444116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dea3d-758c-42e2-b2e9-eb4a8e35b3aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:29.471709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23dea3d-758c-42e2-b2e9-eb4a8e35b3aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:29.476342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 864/1733] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:29.477287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23dea3d-758c-42e2-b2e9-eb4a8e35b3aa service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:29.501018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 870/1734] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:30.394378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547be73-78ba-43e5-87d2-b7906ec18a56 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:30.398077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6547be73-78ba-43e5-87d2-b7906ec18a56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:30.441052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547be73-78ba-43e5-87d2-b7906ec18a56 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:30.456711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:30.459982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:30.460903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:30.461607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] PATH_INFO: `/v3/projects/c9fd9b23df55413aa29f121b9d22df1e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:30.466744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9fd9b23df55413aa29f121b9d22df1e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:30.472180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:30.595723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9fd9b23df55413aa29f121b9d22df1e', 'request_id': 'req-a5586abe-9b9f-4b85-8931-c1374cd8e688'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:30.618188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9fd9b23df55413aa29f121b9d22df1e', 'request_id': 'req-a5586abe-9b9f-4b85-8931-c1374cd8e688'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:30.638469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9fd9b23df55413aa29f121b9d22df1e', 'request_id': 'req-a5586abe-9b9f-4b85-8931-c1374cd8e688'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:30.677140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] The token cache is being invalidate because project c9fd9b23df55413aa29f121b9d22df1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:30.677140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:30.679235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:30.687515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 865/1735] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:30 2022] DELETE /identity/v3/projects/c9fd9b23df55413aa29f121b9d22df1e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:30.688817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dea3d-758c-42e2-b2e9-eb4a8e35b3aa service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:30.702760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dea3d-758c-42e2-b2e9-eb4a8e35b3aa service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:30.828775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:30.845279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:30.847426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:30.849874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:30.850402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] PATH_INFO: `/v3/users/352a3d88037540688f4dab794bddc356` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:30.853341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=352a3d88037540688f4dab794bddc356)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:30.859049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:30.978167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '352a3d88037540688f4dab794bddc356'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:30.979411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '352a3d88037540688f4dab794bddc356'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:31.004866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '352a3d88037540688f4dab794bddc356'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:31.095335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '352a3d88037540688f4dab794bddc356'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:31.099767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-289c66e3-c986-4334-acdf-cb204dcadf55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '352a3d88037540688f4dab794bddc356'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:31.173348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 871/1736] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:30 2022] DELETE /identity/v3/users/352a3d88037540688f4dab794bddc356 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:32.429651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:32.432205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:32.499835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5586abe-9b9f-4b85-8931-c1374cd8e688 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:32.747111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:32.752210 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:32.753154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:32.753594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] PATH_INFO: `/v3/projects/305901ca0f594b4581ab34bb111eef69` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:32.756098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=305901ca0f594b4581ab34bb111eef69)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:32.758513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:32.852494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:32.916806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305901ca0f594b4581ab34bb111eef69', 'request_id': 'req-9a398901-4250-420a-91eb-3c39b88f3f28'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:32.950446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305901ca0f594b4581ab34bb111eef69', 'request_id': 'req-9a398901-4250-420a-91eb-3c39b88f3f28'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:32.995370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305901ca0f594b4581ab34bb111eef69', 'request_id': 'req-9a398901-4250-420a-91eb-3c39b88f3f28'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:33.055165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] The token cache is being invalidate because project 305901ca0f594b4581ab34bb111eef69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:33.056340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:33.061552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9a398901-4250-420a-91eb-3c39b88f3f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:33.072587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 866/1737] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:32 2022] DELETE /identity/v3/projects/305901ca0f594b4581ab34bb111eef69 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:33.077202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e876c8-ba46-4c6a-8c95-07c45a117459 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:33.077744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e876c8-ba46-4c6a-8c95-07c45a117459 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:33.077854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e876c8-ba46-4c6a-8c95-07c45a117459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:33.150843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c0e876c8-ba46-4c6a-8c95-07c45a117459 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:33.154489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c0e876c8-ba46-4c6a-8c95-07c45a117459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:33.385243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 872/1738] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:33.396391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743283a-c316-4f9f-b29b-89fc9a49a6f6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:33.396706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743283a-c316-4f9f-b29b-89fc9a49a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:33.397321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743283a-c316-4f9f-b29b-89fc9a49a6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:33.446240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b743283a-c316-4f9f-b29b-89fc9a49a6f6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:33.449471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b743283a-c316-4f9f-b29b-89fc9a49a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:33.474704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 867/1739] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:33.478128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e876c8-ba46-4c6a-8c95-07c45a117459 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:33.492086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e876c8-ba46-4c6a-8c95-07c45a117459 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:33.506174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48d11b-503e-48f1-abca-3ef446c377b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:33.507180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48d11b-503e-48f1-abca-3ef446c377b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:33.507789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48d11b-503e-48f1-abca-3ef446c377b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:33.509025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48d11b-503e-48f1-abca-3ef446c377b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:33.511805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48d11b-503e-48f1-abca-3ef446c377b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:33.512266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48d11b-503e-48f1-abca-3ef446c377b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:33.515449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48d11b-503e-48f1-abca-3ef446c377b1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:33.548141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97715eb-ae9d-4a8a-ba28-627559e0791a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:33.548423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97715eb-ae9d-4a8a-ba28-627559e0791a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:33.548954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97715eb-ae9d-4a8a-ba28-627559e0791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:33.591391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 873/1740] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:33.595120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48d11b-503e-48f1-abca-3ef446c377b1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:33.602239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f97715eb-ae9d-4a8a-ba28-627559e0791a None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:33.603677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f97715eb-ae9d-4a8a-ba28-627559e0791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:33.646657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48d11b-503e-48f1-abca-3ef446c377b1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:33.657289 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9a748-daa1-4eb5-b759-ff463d5a7528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:33.658825 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a748-daa1-4eb5-b759-ff463d5a7528 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:33.659107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a748-daa1-4eb5-b759-ff463d5a7528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:33.663449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a748-daa1-4eb5-b759-ff463d5a7528 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:33.664293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9a748-daa1-4eb5-b759-ff463d5a7528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:33.666441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9a748-daa1-4eb5-b759-ff463d5a7528 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:33.756895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 874/1741] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:33 2022] POST /identity/v3/projects => generated 393 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:33.761408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9a748-daa1-4eb5-b759-ff463d5a7528 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:33.800277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 868/1742] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:34:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:33.897369 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:33.975842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9a748-daa1-4eb5-b759-ff463d5a7528 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:34.004540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e393f-77ff-4288-bcb2-f523dce6e54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:34.005570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e393f-77ff-4288-bcb2-f523dce6e54d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:34.005979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e393f-77ff-4288-bcb2-f523dce6e54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:34.006211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e393f-77ff-4288-bcb2-f523dce6e54d admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:34.007104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e393f-77ff-4288-bcb2-f523dce6e54d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:34.012796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e393f-77ff-4288-bcb2-f523dce6e54d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:34.074079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 875/1743] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:33 2022] POST /identity/v3/users => generated 297 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:34.078359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97715eb-ae9d-4a8a-ba28-627559e0791a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:34.113417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97715eb-ae9d-4a8a-ba28-627559e0791a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:34.146045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38970711-7c20-4197-962a-a586a9a65d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:34.146928 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38970711-7c20-4197-962a-a586a9a65d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:34.147210 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38970711-7c20-4197-962a-a586a9a65d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:34.147721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38970711-7c20-4197-962a-a586a9a65d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:34.148713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38970711-7c20-4197-962a-a586a9a65d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:34.153227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38970711-7c20-4197-962a-a586a9a65d1d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:34.168452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e393f-77ff-4288-bcb2-f523dce6e54d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:34.170735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-283e393f-77ff-4288-bcb2-f523dce6e54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:34.210317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 869/1744] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:34.211585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38970711-7c20-4197-962a-a586a9a65d1d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:34.243227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38970711-7c20-4197-962a-a586a9a65d1d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:34.269139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c74258-358f-4944-b9b9-5bdb9bb27d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:34.269839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c74258-358f-4944-b9b9-5bdb9bb27d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:34.270125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c74258-358f-4944-b9b9-5bdb9bb27d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:34.270408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c74258-358f-4944-b9b9-5bdb9bb27d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:34.271137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c74258-358f-4944-b9b9-5bdb9bb27d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:34.281783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c74258-358f-4944-b9b9-5bdb9bb27d72 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:34.316104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 870/1745] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:34.332487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c74258-358f-4944-b9b9-5bdb9bb27d72 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:34.385880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c74258-358f-4944-b9b9-5bdb9bb27d72 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:34.411184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f15fb-dbe6-40d6-942e-8e23d380364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:34.412756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f15fb-dbe6-40d6-942e-8e23d380364d admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:34.412756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f15fb-dbe6-40d6-942e-8e23d380364d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:34.424414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f15fb-dbe6-40d6-942e-8e23d380364d admin admin] PATH_INFO: `/v3/projects/441248be0f8f4f79b1411411e24cb636/users/ba7778ed8a414ce084cac7f882c4350a/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:34.480562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e393f-77ff-4288-bcb2-f523dce6e54d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:34.489509 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f15fb-dbe6-40d6-942e-8e23d380364d admin admin] RBAC: Authorizing `identity:create_grant(project_id=441248be0f8f4f79b1411411e24cb636, user_id=ba7778ed8a414ce084cac7f882c4350a, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:34.498037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266f15fb-dbe6-40d6-942e-8e23d380364d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:34.502761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6120ac-f43c-4927-b754-248876022383 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:34.504055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6120ac-f43c-4927-b754-248876022383 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:34.504542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6120ac-f43c-4927-b754-248876022383 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:34.504841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6120ac-f43c-4927-b754-248876022383 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:34.541536 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 871/1746] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:34 2022] PUT /identity/v3/projects/441248be0f8f4f79b1411411e24cb636/users/ba7778ed8a414ce084cac7f882c4350a/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:34.554973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65c5e5-4abe-45b8-b70b-95a54f03ae08 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:34.555522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65c5e5-4abe-45b8-b70b-95a54f03ae08 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:34.555806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65c5e5-4abe-45b8-b70b-95a54f03ae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:34.632989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0e65c5e5-4abe-45b8-b70b-95a54f03ae08 None None] MFA Rules not processed for user `ba7778ed8a414ce084cac7f882c4350a`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:34.634299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0e65c5e5-4abe-45b8-b70b-95a54f03ae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:34.702254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6120ac-f43c-4927-b754-248876022383 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:34.704022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6120ac-f43c-4927-b754-248876022383 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:34.735783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 876/1747] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:34:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:34:34.859759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 872/1748] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:34.861079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6120ac-f43c-4927-b754-248876022383 service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:34.887323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6120ac-f43c-4927-b754-248876022383 service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:35.074503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee9595-ccb6-4a9a-9cab-f7879d0dca14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:35.075518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee9595-ccb6-4a9a-9cab-f7879d0dca14 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:35.075970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee9595-ccb6-4a9a-9cab-f7879d0dca14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:35.076390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee9595-ccb6-4a9a-9cab-f7879d0dca14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:35.077243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee9595-ccb6-4a9a-9cab-f7879d0dca14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:35.079367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee9595-ccb6-4a9a-9cab-f7879d0dca14 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:35.124210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:35.163745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 877/1749] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:34 2022] POST /identity/v3/projects => generated 393 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:35.169767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65c5e5-4abe-45b8-b70b-95a54f03ae08 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:35.179905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:35.202155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65c5e5-4abe-45b8-b70b-95a54f03ae08 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:35.399329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6386-b7b0-4d89-8d6a-498f4176d861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:35.399988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6386-b7b0-4d89-8d6a-498f4176d861 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:35.400400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6386-b7b0-4d89-8d6a-498f4176d861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:35.400684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f6386-b7b0-4d89-8d6a-498f4176d861 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:35.401393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f6386-b7b0-4d89-8d6a-498f4176d861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:35.403420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f6386-b7b0-4d89-8d6a-498f4176d861 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:35.456472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 873/1750] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:35 2022] POST /identity/v3/users => generated 303 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:35.461258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee9595-ccb6-4a9a-9cab-f7879d0dca14 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:35.474415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee9595-ccb6-4a9a-9cab-f7879d0dca14 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:35.493239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d4af8-b4bb-4751-a8f9-879ff8f03ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:35.494217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d4af8-b4bb-4751-a8f9-879ff8f03ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:35.495076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d4af8-b4bb-4751-a8f9-879ff8f03ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:35.495658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d4af8-b4bb-4751-a8f9-879ff8f03ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:35.497350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d4af8-b4bb-4751-a8f9-879ff8f03ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:35.501512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d4af8-b4bb-4751-a8f9-879ff8f03ae5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:35.539760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 878/1751] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:35.544283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6386-b7b0-4d89-8d6a-498f4176d861 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:35.555800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f6386-b7b0-4d89-8d6a-498f4176d861 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:35.592310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbddf6-53ca-49bc-a161-758a6941baab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:35.594147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbddf6-53ca-49bc-a161-758a6941baab admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:35.604038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbddf6-53ca-49bc-a161-758a6941baab admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:35.605050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbddf6-53ca-49bc-a161-758a6941baab admin admin] PATH_INFO: `/v3/projects/9a3c2ce52b1849979f0748e6597d195a/users/0db908f3bafc4d93b4b8f0493c6a1fbf/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:35.663627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbddf6-53ca-49bc-a161-758a6941baab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3c2ce52b1849979f0748e6597d195a, user_id=0db908f3bafc4d93b4b8f0493c6a1fbf, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:35.672019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbddf6-53ca-49bc-a161-758a6941baab admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:35.712212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 874/1752] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:35 2022] PUT /identity/v3/projects/9a3c2ce52b1849979f0748e6597d195a/users/0db908f3bafc4d93b4b8f0493c6a1fbf/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:35.716433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d4af8-b4bb-4751-a8f9-879ff8f03ae5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:35.727084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d4af8-b4bb-4751-a8f9-879ff8f03ae5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:35.934373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1e8f-ec06-41d8-ad57-bb0b86bdb3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:35.937246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1e8f-ec06-41d8-ad57-bb0b86bdb3df admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:35.937689 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1e8f-ec06-41d8-ad57-bb0b86bdb3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:35.938896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1e8f-ec06-41d8-ad57-bb0b86bdb3df admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:35.941068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1e8f-ec06-41d8-ad57-bb0b86bdb3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:35.947996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1e8f-ec06-41d8-ad57-bb0b86bdb3df admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:36.011400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 879/1753] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:36.012236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbddf6-53ca-49bc-a161-758a6941baab admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:36.047707 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbddf6-53ca-49bc-a161-758a6941baab admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:36.076814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34668fee-5401-455d-a3f0-bcb9cac9bc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:36.079365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668fee-5401-455d-a3f0-bcb9cac9bc63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:36.080613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668fee-5401-455d-a3f0-bcb9cac9bc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:36.081774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668fee-5401-455d-a3f0-bcb9cac9bc63 admin admin] PATH_INFO: `/v3/projects/9a3c2ce52b1849979f0748e6597d195a/users/0db908f3bafc4d93b4b8f0493c6a1fbf/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:36.090982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34668fee-5401-455d-a3f0-bcb9cac9bc63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3c2ce52b1849979f0748e6597d195a, user_id=0db908f3bafc4d93b4b8f0493c6a1fbf, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:36.134466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34668fee-5401-455d-a3f0-bcb9cac9bc63 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:36.207371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 875/1754] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:36 2022] PUT /identity/v3/projects/9a3c2ce52b1849979f0748e6597d195a/users/0db908f3bafc4d93b4b8f0493c6a1fbf/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:36.215888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29214717-0b19-497f-9365-f3e0b6dc0e32 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:36.216464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29214717-0b19-497f-9365-f3e0b6dc0e32 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:36.216958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29214717-0b19-497f-9365-f3e0b6dc0e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:36.306374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-29214717-0b19-497f-9365-f3e0b6dc0e32 None None] MFA Rules not processed for user `0db908f3bafc4d93b4b8f0493c6a1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:36.310912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29214717-0b19-497f-9365-f3e0b6dc0e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:36.531835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e92c24-7479-459d-b5df-932aefbe3ac2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:36.532234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e92c24-7479-459d-b5df-932aefbe3ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:36.532523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e92c24-7479-459d-b5df-932aefbe3ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:36.578139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 880/1755] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:36.582576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29214717-0b19-497f-9365-f3e0b6dc0e32 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:36.589639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29214717-0b19-497f-9365-f3e0b6dc0e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:36.591727 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-20e92c24-7479-459d-b5df-932aefbe3ac2 None None] MFA Rules not processed for user `d67be26446324c91b956e9d930e04b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:36.593507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-20e92c24-7479-459d-b5df-932aefbe3ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:36.721625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 876/1756] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Jul 2 03:34:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:36.723014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e92c24-7479-459d-b5df-932aefbe3ac2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:36.727133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-20e92c24-7479-459d-b5df-932aefbe3ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:36.880545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29214717-0b19-497f-9365-f3e0b6dc0e32 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:36.901040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e92c24-7479-459d-b5df-932aefbe3ac2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:36.908697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:36.915268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbf528-09a9-4e99-a1f9-fd66f896ed98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:36.916217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:36.916589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:36.916854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbf528-09a9-4e99-a1f9-fd66f896ed98 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:36.917395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbf528-09a9-4e99-a1f9-fd66f896ed98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:36.917894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbf528-09a9-4e99-a1f9-fd66f896ed98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:36.919008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] PATH_INFO: `/v3/users/7485a53bf94b424ca45a9be4d0061e5e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:36.920869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7485a53bf94b424ca45a9be4d0061e5e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:36.927429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:36.938496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbf528-09a9-4e99-a1f9-fd66f896ed98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:36.940382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbf528-09a9-4e99-a1f9-fd66f896ed98 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:36.953719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:36.965220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 877/1757] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:36.966886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fec5a-946f-4add-bc1d-e5e0cfff5512 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:36.970631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fec5a-946f-4add-bc1d-e5e0cfff5512 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:36.971190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fec5a-946f-4add-bc1d-e5e0cfff5512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:37.008136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:37.028154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3a0fec5a-946f-4add-bc1d-e5e0cfff5512 None None] MFA Rules not processed for user `868e2f0e822b430981369d414eee8e53`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:37.029793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a0fec5a-946f-4add-bc1d-e5e0cfff5512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:37.049656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7485a53bf94b424ca45a9be4d0061e5e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:37.050920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7485a53bf94b424ca45a9be4d0061e5e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:37.074579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7485a53bf94b424ca45a9be4d0061e5e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:37.131203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7485a53bf94b424ca45a9be4d0061e5e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:37.132609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7485a53bf94b424ca45a9be4d0061e5e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:37.212443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 881/1758] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:36 2022] DELETE /identity/v3/users/7485a53bf94b424ca45a9be4d0061e5e => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:37.214330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:37.218308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 878/1759] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:37.219764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061a38d-e6cd-42f8-ad53-ccb08bd719eb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:37.220038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:37.220150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061a38d-e6cd-42f8-ad53-ccb08bd719eb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:37.220326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061a38d-e6cd-42f8-ad53-ccb08bd719eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:37.272339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4061a38d-e6cd-42f8-ad53-ccb08bd719eb None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:37.273844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4061a38d-e6cd-42f8-ad53-ccb08bd719eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:37.466837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b94bd5-9aa2-48dc-86ab-d120010eea83 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:37.483316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:37.484676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:37.485243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:37.485897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:37.496822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:37.520870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:37.522847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:37.547821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 882/1760] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:37.548945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:37.551542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:37.564532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 879/1761] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:37.653898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869a626-0988-4aa0-8c80-6acecbbd54a0 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:37.661862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:37.663407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:37.664008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:37.664504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:37.665227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061a38d-e6cd-42f8-ad53-ccb08bd719eb None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:37.667499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:37.684653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061a38d-e6cd-42f8-ad53-ccb08bd719eb None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:37.691358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8fe5a-c799-4575-bd91-8bbca53b208a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:37.692033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fe5a-c799-4575-bd91-8bbca53b208a service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:37.692319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fe5a-c799-4575-bd91-8bbca53b208a service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:37.692598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fe5a-c799-4575-bd91-8bbca53b208a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:37.712487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8fe5a-c799-4575-bd91-8bbca53b208a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:37.713998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8fe5a-c799-4575-bd91-8bbca53b208a service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:37.737529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 880/1762] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:37.792111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:37.835225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:37.837169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:37.879712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 883/1763] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:39.022281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8fe5a-c799-4575-bd91-8bbca53b208a service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:39.024569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2bf8fe5a-c799-4575-bd91-8bbca53b208a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:39.056160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8fe5a-c799-4575-bd91-8bbca53b208a service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:39.062830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:39.063838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:39.064389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:39.064898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] PATH_INFO: `/v3/users/fdb0aef8e7ff4e6089d4ea780a85b0fe` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:39.067774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb0aef8e7ff4e6089d4ea780a85b0fe)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:39.074895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:39.115008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:39.118686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:39.160677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c78c0-c40c-403e-9f63-fdab41efd3f9 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:39.178572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:39.179790 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:39.180128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:39.180639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] PATH_INFO: `/v3/projects/ced4717b9d1d48e0b3371d50b81a3770` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:39.183918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ced4717b9d1d48e0b3371d50b81a3770)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:39.189355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:39.309498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ced4717b9d1d48e0b3371d50b81a3770', 'request_id': 'req-20118d57-efbe-4be9-a954-56372897744e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.325542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ced4717b9d1d48e0b3371d50b81a3770', 'request_id': 'req-20118d57-efbe-4be9-a954-56372897744e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.325769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb0aef8e7ff4e6089d4ea780a85b0fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.327358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb0aef8e7ff4e6089d4ea780a85b0fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.335249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ced4717b9d1d48e0b3371d50b81a3770', 'request_id': 'req-20118d57-efbe-4be9-a954-56372897744e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.356934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb0aef8e7ff4e6089d4ea780a85b0fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.358181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] The token cache is being invalidate because project ced4717b9d1d48e0b3371d50b81a3770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:39.358792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.358923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb0aef8e7ff4e6089d4ea780a85b0fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.360025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.363927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 884/1764] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:39 2022] DELETE /identity/v3/projects/ced4717b9d1d48e0b3371d50b81a3770 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:39.368063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:39.371255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:39.428142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5393dc26-1904-4a22-a0d2-7f315657b603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb0aef8e7ff4e6089d4ea780a85b0fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.488948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 881/1765] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:39 2022] DELETE /identity/v3/users/fdb0aef8e7ff4e6089d4ea780a85b0fe => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:39.490555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0209c-5c3c-49b4-bd54-1c6ba69bf051 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:39.490852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0209c-5c3c-49b4-bd54-1c6ba69bf051 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:39.491275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0209c-5c3c-49b4-bd54-1c6ba69bf051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:39.515003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20118d57-efbe-4be9-a954-56372897744e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:39.542932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fba0209c-5c3c-49b4-bd54-1c6ba69bf051 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:39.549057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fba0209c-5c3c-49b4-bd54-1c6ba69bf051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:39.563084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:39.615994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:39.617109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:39.618368 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:39.619612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] PATH_INFO: `/v3/users/98b69773f2b44c39913a34d884a8c362` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:39.624604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b69773f2b44c39913a34d884a8c362)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:39.627173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:39.652711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:39.656604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 882/1766] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:39.674000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0209c-5c3c-49b4-bd54-1c6ba69bf051 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:39.675912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fba0209c-5c3c-49b4-bd54-1c6ba69bf051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:39.739825 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b69773f2b44c39913a34d884a8c362'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.743746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b69773f2b44c39913a34d884a8c362'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.765464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b69773f2b44c39913a34d884a8c362'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.796818 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0209c-5c3c-49b4-bd54-1c6ba69bf051 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:39.809121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b69773f2b44c39913a34d884a8c362'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.809353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:39.810326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:39.810619 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:39.810907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:39.812401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b69773f2b44c39913a34d884a8c362'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:39.818173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:39.838089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 885/1767] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:39 2022] DELETE /identity/v3/users/98b69773f2b44c39913a34d884a8c362 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:39.858076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:39.862019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:39.884046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 883/1768] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:40.693579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:40.695219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:40.730003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:40.732550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:40.760441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2a2dc-6b45-4496-b04d-4b4f66e9feb7 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:40.813730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac53591-5fa1-4d5b-afa4-c370c825c751 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:40.825245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:40.829230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:40.829434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:40.829644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] PATH_INFO: `/v3/projects/9aaaef2d19204bc8909aee5b2b330882` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:40.834425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aaaef2d19204bc8909aee5b2b330882)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:40.840991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:40.971252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aaaef2d19204bc8909aee5b2b330882', 'request_id': 'req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:40.994461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aaaef2d19204bc8909aee5b2b330882', 'request_id': 'req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.028731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aaaef2d19204bc8909aee5b2b330882', 'request_id': 'req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.037916 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:41.038629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:41.038918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:41.039198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] PATH_INFO: `/v3/projects/33eaf69c5a8f434a98fb0940ad8464bf` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:41.040896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33eaf69c5a8f434a98fb0940ad8464bf)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:41.043898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:41.057371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] The token cache is being invalidate because project 9aaaef2d19204bc8909aee5b2b330882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:41.057739 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.059666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa2ef1a7-87b1-44d7-a646-4db012ea72b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.066283 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 886/1769] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:40 2022] DELETE /identity/v3/projects/9aaaef2d19204bc8909aee5b2b330882 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:41.068785 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8063a-707e-401f-912a-fc8ad4c25c66 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:41.069070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8063a-707e-401f-912a-fc8ad4c25c66 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:41.069344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8063a-707e-401f-912a-fc8ad4c25c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:41.150806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7ad8063a-707e-401f-912a-fc8ad4c25c66 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:41.158406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7ad8063a-707e-401f-912a-fc8ad4c25c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:41.251401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:41.259685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33eaf69c5a8f434a98fb0940ad8464bf', 'request_id': 'req-be4e2eaa-05f7-441e-8e44-703506d9fb44'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.283866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33eaf69c5a8f434a98fb0940ad8464bf', 'request_id': 'req-be4e2eaa-05f7-441e-8e44-703506d9fb44'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.286201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:41.305254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33eaf69c5a8f434a98fb0940ad8464bf', 'request_id': 'req-be4e2eaa-05f7-441e-8e44-703506d9fb44'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.346261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] The token cache is being invalidate because project 33eaf69c5a8f434a98fb0940ad8464bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:41.347284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.350284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.360472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 884/1770] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:40 2022] DELETE /identity/v3/projects/33eaf69c5a8f434a98fb0940ad8464bf => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:41.361783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:41.362478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 887/1771] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:41.365606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e021a8-f34a-4006-8cd2-3e1cfff8685f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:41.366256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e021a8-f34a-4006-8cd2-3e1cfff8685f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:41.366693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e021a8-f34a-4006-8cd2-3e1cfff8685f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:41.368541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:41.393380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-63e021a8-f34a-4006-8cd2-3e1cfff8685f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:41.394717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-63e021a8-f34a-4006-8cd2-3e1cfff8685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:41.626932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e2eaa-05f7-441e-8e44-703506d9fb44 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:41.634446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:41.635743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:41.636214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:41.636626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] PATH_INFO: `/v3/users/0bceddefdd3d4b59a03ca15973c84e48` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:41.638303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bceddefdd3d4b59a03ca15973c84e48)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:41.641822 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:41.692638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 888/1772] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:41.703385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e021a8-f34a-4006-8cd2-3e1cfff8685f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:41.746044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e021a8-f34a-4006-8cd2-3e1cfff8685f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:41.768297 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bceddefdd3d4b59a03ca15973c84e48'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.769259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bceddefdd3d4b59a03ca15973c84e48'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.789071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bceddefdd3d4b59a03ca15973c84e48'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.789450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:41.790520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:41.790520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:41.790819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:41.791536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:41.791846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:41.794111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:41.795051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bceddefdd3d4b59a03ca15973c84e48'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.859310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 889/1773] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:41.860695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:41.876577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:41.881653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bceddefdd3d4b59a03ca15973c84e48'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:41.936410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 885/1774] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:41 2022] DELETE /identity/v3/users/0bceddefdd3d4b59a03ca15973c84e48 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:41.972300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a5320-24b1-4ccd-b022-03c615826c9b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:42.224319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c7d99-f596-4c71-a011-39b6c3a8fd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:42.225755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c7d99-f596-4c71-a011-39b6c3a8fd5c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:42.226293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c7d99-f596-4c71-a011-39b6c3a8fd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:42.227160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c7d99-f596-4c71-a011-39b6c3a8fd5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:42.229536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c7d99-f596-4c71-a011-39b6c3a8fd5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:42.233001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c7d99-f596-4c71-a011-39b6c3a8fd5c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:42.311313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 890/1775] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:41 2022] POST /identity/v3/projects => generated 363 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:42.316778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:42.337301 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4766903-c4a4-4a30-af03-3250315fb265 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:42.578421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b99d46-64dc-49c3-bf34-3f5b3d80fefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:42.579678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b99d46-64dc-49c3-bf34-3f5b3d80fefd admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:42.579939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b99d46-64dc-49c3-bf34-3f5b3d80fefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:42.580218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b99d46-64dc-49c3-bf34-3f5b3d80fefd admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:42.580949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b99d46-64dc-49c3-bf34-3f5b3d80fefd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:42.582973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b99d46-64dc-49c3-bf34-3f5b3d80fefd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:42.606098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:42.631486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 886/1776] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:42 2022] POST /identity/v3/users => generated 282 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:42.634228 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c7d99-f596-4c71-a011-39b6c3a8fd5c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:42.647109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c7d99-f596-4c71-a011-39b6c3a8fd5c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:42.678913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040743c-e118-4bc2-bdab-aa5a618320fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:42.684494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040743c-e118-4bc2-bdab-aa5a618320fa admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:42.684967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040743c-e118-4bc2-bdab-aa5a618320fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:42.685398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040743c-e118-4bc2-bdab-aa5a618320fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:42.686246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040743c-e118-4bc2-bdab-aa5a618320fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:42.690890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040743c-e118-4bc2-bdab-aa5a618320fa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:42.742595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 891/1777] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:42.743439 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b99d46-64dc-49c3-bf34-3f5b3d80fefd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:42.761160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b99d46-64dc-49c3-bf34-3f5b3d80fefd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:42.783086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f683a-38a3-41d1-8270-b92b0b024c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:42.783794 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f683a-38a3-41d1-8270-b92b0b024c65 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:42.784077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f683a-38a3-41d1-8270-b92b0b024c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:42.784524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f683a-38a3-41d1-8270-b92b0b024c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:42.785407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f683a-38a3-41d1-8270-b92b0b024c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:42.787191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f683a-38a3-41d1-8270-b92b0b024c65 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:42.824662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 887/1778] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:34:42.825886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040743c-e118-4bc2-bdab-aa5a618320fa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:42.841881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040743c-e118-4bc2-bdab-aa5a618320fa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:42.855761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61893bf6-0e91-4ee2-9436-9d0c6d6cc59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:42.857028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61893bf6-0e91-4ee2-9436-9d0c6d6cc59e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:42.857470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61893bf6-0e91-4ee2-9436-9d0c6d6cc59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:42.857749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61893bf6-0e91-4ee2-9436-9d0c6d6cc59e admin admin] PATH_INFO: `/v3/projects/8a8989289a0e4cfb9c60f0e07ef3b09d/users/95c8d51c628444238bedd8931538cc54/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:42.939562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61893bf6-0e91-4ee2-9436-9d0c6d6cc59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8989289a0e4cfb9c60f0e07ef3b09d, user_id=95c8d51c628444238bedd8931538cc54, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:42.942405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61893bf6-0e91-4ee2-9436-9d0c6d6cc59e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:42.966431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5351cdf-6d25-4072-9306-fc7e6091b816 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:42.967020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5351cdf-6d25-4072-9306-fc7e6091b816 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:42.967746 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5351cdf-6d25-4072-9306-fc7e6091b816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:42.981698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:43.006205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 892/1779] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:42 2022] PUT /identity/v3/projects/8a8989289a0e4cfb9c60f0e07ef3b09d/users/95c8d51c628444238bedd8931538cc54/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:43.017190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4156d6-0fe7-4e7c-965f-56e0382d8cc6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:43.017484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4156d6-0fe7-4e7c-965f-56e0382d8cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:43.017761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4156d6-0fe7-4e7c-965f-56e0382d8cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:43.058349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b5351cdf-6d25-4072-9306-fc7e6091b816 None None] MFA Rules not processed for user `112d731047a64efc8e9143c4d471a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:43.091086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b5351cdf-6d25-4072-9306-fc7e6091b816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:43.134929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7f4156d6-0fe7-4e7c-965f-56e0382d8cc6 None None] MFA Rules not processed for user `95c8d51c628444238bedd8931538cc54`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:43.137330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7f4156d6-0fe7-4e7c-965f-56e0382d8cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:43.321506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 888/1780] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:43.427922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 893/1781] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:43.432778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5351cdf-6d25-4072-9306-fc7e6091b816 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:43.592611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4156d6-0fe7-4e7c-965f-56e0382d8cc6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:43.594597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7f4156d6-0fe7-4e7c-965f-56e0382d8cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:43.732213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5351cdf-6d25-4072-9306-fc7e6091b816 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:43.750055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04156521-f420-44e0-9ede-845da6018bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:43.751087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04156521-f420-44e0-9ede-845da6018bd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:43.751601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04156521-f420-44e0-9ede-845da6018bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:43.752041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04156521-f420-44e0-9ede-845da6018bd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:43.753062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04156521-f420-44e0-9ede-845da6018bd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:43.756077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04156521-f420-44e0-9ede-845da6018bd8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:43.764874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4156d6-0fe7-4e7c-965f-56e0382d8cc6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:43.774029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:43.774944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:43.775222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:43.778086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] PATH_INFO: `/v3/users/ba7778ed8a414ce084cac7f882c4350a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:43.782263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba7778ed8a414ce084cac7f882c4350a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:43.786439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:43.811416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 889/1782] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:43 2022] POST /identity/v3/projects => generated 365 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:43.814378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04156521-f420-44e0-9ede-845da6018bd8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:43.927940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba7778ed8a414ce084cac7f882c4350a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:43.931321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba7778ed8a414ce084cac7f882c4350a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:43.965499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba7778ed8a414ce084cac7f882c4350a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:44.018843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba7778ed8a414ce084cac7f882c4350a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:44.019614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-433635fe-2863-422e-beb6-00350c513c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba7778ed8a414ce084cac7f882c4350a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:44.065177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04156521-f420-44e0-9ede-845da6018bd8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:44.069223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 894/1783] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:43 2022] DELETE /identity/v3/users/ba7778ed8a414ce084cac7f882c4350a => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:44.079649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dd0d7-b3c7-4b9f-b394-f34acde7678f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:44.080063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dd0d7-b3c7-4b9f-b394-f34acde7678f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:44.080239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dd0d7-b3c7-4b9f-b394-f34acde7678f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:44.157435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-417dd0d7-b3c7-4b9f-b394-f34acde7678f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:44.163817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:44.171133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-417dd0d7-b3c7-4b9f-b394-f34acde7678f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:44.198023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:44.458160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 895/1784] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:44.461998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7683d76-18bd-450e-ab0f-368a9415dec8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:44.462511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7683d76-18bd-450e-ab0f-368a9415dec8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:44.463119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7683d76-18bd-450e-ab0f-368a9415dec8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:44.465283 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 896/1785] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:34:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:44.497798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744cae3-a422-4f09-8ae7-8e50126d6243 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:44.498605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f223714-9e2a-4402-9736-b423fcf61314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:44.499001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744cae3-a422-4f09-8ae7-8e50126d6243 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:44.499557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744cae3-a422-4f09-8ae7-8e50126d6243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:44.499675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f223714-9e2a-4402-9736-b423fcf61314 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:44.500025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f223714-9e2a-4402-9736-b423fcf61314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:44.500378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f223714-9e2a-4402-9736-b423fcf61314 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:44.501174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f223714-9e2a-4402-9736-b423fcf61314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:44.503213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f223714-9e2a-4402-9736-b423fcf61314 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:44.567730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 890/1786] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:43 2022] POST /identity/v3/users => generated 289 bytes in 753 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:44.568524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f223714-9e2a-4402-9736-b423fcf61314 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:44.572208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6f223714-9e2a-4402-9736-b423fcf61314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:44.591807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f223714-9e2a-4402-9736-b423fcf61314 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:44.608097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed70c59-bccb-4639-9a0c-9ea68a303423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:44.609144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed70c59-bccb-4639-9a0c-9ea68a303423 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:44.609144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed70c59-bccb-4639-9a0c-9ea68a303423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:44.609409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed70c59-bccb-4639-9a0c-9ea68a303423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:44.610157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed70c59-bccb-4639-9a0c-9ea68a303423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:44.611825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed70c59-bccb-4639-9a0c-9ea68a303423 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:44.638252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3744cae3-a422-4f09-8ae7-8e50126d6243 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:44.642519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 891/1787] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:44.643736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed70c59-bccb-4639-9a0c-9ea68a303423 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:44.651717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3744cae3-a422-4f09-8ae7-8e50126d6243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:44.669977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed70c59-bccb-4639-9a0c-9ea68a303423 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:44.680300 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27febdf5-6f26-477b-8d2f-5af0a90f9627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:44.681225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27febdf5-6f26-477b-8d2f-5af0a90f9627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:44.681504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27febdf5-6f26-477b-8d2f-5af0a90f9627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:44.681778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27febdf5-6f26-477b-8d2f-5af0a90f9627 admin admin] PATH_INFO: `/v3/projects/eef87dfe92b2434695779feddfafe39c/users/f31c40f46aa246448a5d4e4bbe66bd08/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:44.739355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27febdf5-6f26-477b-8d2f-5af0a90f9627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef87dfe92b2434695779feddfafe39c, user_id=f31c40f46aa246448a5d4e4bbe66bd08, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:44.745844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27febdf5-6f26-477b-8d2f-5af0a90f9627 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:44.780793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 892/1788] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:44 2022] PUT /identity/v3/projects/eef87dfe92b2434695779feddfafe39c/users/f31c40f46aa246448a5d4e4bbe66bd08/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:44.781925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27febdf5-6f26-477b-8d2f-5af0a90f9627 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:44.849109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 897/1789] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:34:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 02 03:34:44.925828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27febdf5-6f26-477b-8d2f-5af0a90f9627 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:44.935747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca05b84-be55-4155-8df0-8a6f204d5726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:44.937083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca05b84-be55-4155-8df0-8a6f204d5726 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:44.937557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca05b84-be55-4155-8df0-8a6f204d5726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:44.937981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca05b84-be55-4155-8df0-8a6f204d5726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:44.939421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca05b84-be55-4155-8df0-8a6f204d5726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:44.941909 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca05b84-be55-4155-8df0-8a6f204d5726 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:44.958254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 893/1790] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:44.962421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744cae3-a422-4f09-8ae7-8e50126d6243 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:44.979458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744cae3-a422-4f09-8ae7-8e50126d6243 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:44.994922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d281-af24-4c20-a8b1-56b605a5978e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:44.997324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d281-af24-4c20-a8b1-56b605a5978e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:44.997619 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d281-af24-4c20-a8b1-56b605a5978e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:44.997901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0d281-af24-4c20-a8b1-56b605a5978e admin admin] PATH_INFO: `/v3/projects/eef87dfe92b2434695779feddfafe39c/users/f31c40f46aa246448a5d4e4bbe66bd08/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:45.003733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0d281-af24-4c20-a8b1-56b605a5978e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef87dfe92b2434695779feddfafe39c, user_id=f31c40f46aa246448a5d4e4bbe66bd08, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:45.009034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0d281-af24-4c20-a8b1-56b605a5978e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:45.043789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 898/1791] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:44 2022] PUT /identity/v3/projects/eef87dfe92b2434695779feddfafe39c/users/f31c40f46aa246448a5d4e4bbe66bd08/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:45.055509 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390374ed-9bf2-413b-9550-142eb9d41107 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:45.055952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390374ed-9bf2-413b-9550-142eb9d41107 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:45.056241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390374ed-9bf2-413b-9550-142eb9d41107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:45.107086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d281-af24-4c20-a8b1-56b605a5978e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:45.110274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5ae0d281-af24-4c20-a8b1-56b605a5978e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:45.152220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-390374ed-9bf2-413b-9550-142eb9d41107 None None] MFA Rules not processed for user `f31c40f46aa246448a5d4e4bbe66bd08`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:45.159420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-390374ed-9bf2-413b-9550-142eb9d41107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:45.386789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0d281-af24-4c20-a8b1-56b605a5978e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:45.387156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:45.399307 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:45.401007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:45.401293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:45.401580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] PATH_INFO: `/v3/projects/24f7faf54dd447b1a8ed0b8ada9004c4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:45.409503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f7faf54dd447b1a8ed0b8ada9004c4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:45.411648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:45.415071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 894/1792] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:45.416067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390374ed-9bf2-413b-9550-142eb9d41107 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:45.420963 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-390374ed-9bf2-413b-9550-142eb9d41107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:45.529420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:45.577896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f7faf54dd447b1a8ed0b8ada9004c4', 'request_id': 'req-ce5189be-3d1e-4a88-a786-fc5e17389549'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:45.593786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f7faf54dd447b1a8ed0b8ada9004c4', 'request_id': 'req-ce5189be-3d1e-4a88-a786-fc5e17389549'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:45.609966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390374ed-9bf2-413b-9550-142eb9d41107 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:45.611267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f7faf54dd447b1a8ed0b8ada9004c4', 'request_id': 'req-ce5189be-3d1e-4a88-a786-fc5e17389549'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:45.641151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] The token cache is being invalidate because project 24f7faf54dd447b1a8ed0b8ada9004c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:45.642389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:45.644969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:45.653315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 899/1793] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:45 2022] DELETE /identity/v3/projects/24f7faf54dd447b1a8ed0b8ada9004c4 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:45.654575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:45.661097 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:45.750921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba9253-758c-465f-a1f3-b78f0c29fa2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:45.751623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba9253-758c-465f-a1f3-b78f0c29fa2d service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:45.754220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba9253-758c-465f-a1f3-b78f0c29fa2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:45.756202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba9253-758c-465f-a1f3-b78f0c29fa2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:45.775245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-feba9253-758c-465f-a1f3-b78f0c29fa2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:45.783286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5189be-3d1e-4a88-a786-fc5e17389549 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:45.788489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7846b65-f1ae-43c2-9e0e-fc1076608d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:45.789169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7846b65-f1ae-43c2-9e0e-fc1076608d8e service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:45.789500 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7846b65-f1ae-43c2-9e0e-fc1076608d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:45.789778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7846b65-f1ae-43c2-9e0e-fc1076608d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:45.791839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e7846b65-f1ae-43c2-9e0e-fc1076608d8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:45.896978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7846b65-f1ae-43c2-9e0e-fc1076608d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:45.898885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7846b65-f1ae-43c2-9e0e-fc1076608d8e service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:45.913898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 900/1794] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:45.915502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd4794-1d6d-4941-9d1a-3f6c74ccb757 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:45.915867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd4794-1d6d-4941-9d1a-3f6c74ccb757 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:45.916148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd4794-1d6d-4941-9d1a-3f6c74ccb757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:45.949138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba9253-758c-465f-a1f3-b78f0c29fa2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:45.951038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba9253-758c-465f-a1f3-b78f0c29fa2d service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:45.959828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d4cd4794-1d6d-4941-9d1a-3f6c74ccb757 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:45.961417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d4cd4794-1d6d-4941-9d1a-3f6c74ccb757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:45.965439 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 895/1795] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:45.990395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 901/1796] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:45.991782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f17cae-c0e9-48bf-b53e-ea709807a713 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:45.992208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f17cae-c0e9-48bf-b53e-ea709807a713 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:45.992412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f17cae-c0e9-48bf-b53e-ea709807a713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:46.017776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-04f17cae-c0e9-48bf-b53e-ea709807a713 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:46.019277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-04f17cae-c0e9-48bf-b53e-ea709807a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:46.034786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd4794-1d6d-4941-9d1a-3f6c74ccb757 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:46.046535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd4794-1d6d-4941-9d1a-3f6c74ccb757 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:46.047563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 896/1797] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:46.052282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54349646-5fd6-4578-914f-4d2cc673af5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:46.052978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54349646-5fd6-4578-914f-4d2cc673af5b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:46.053267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54349646-5fd6-4578-914f-4d2cc673af5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:46.053553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54349646-5fd6-4578-914f-4d2cc673af5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:46.054570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54349646-5fd6-4578-914f-4d2cc673af5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:46.054882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54349646-5fd6-4578-914f-4d2cc673af5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:46.056581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54349646-5fd6-4578-914f-4d2cc673af5b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:46.072449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 902/1798] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:34:46.076782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f17cae-c0e9-48bf-b53e-ea709807a713 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:46.089199 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f17cae-c0e9-48bf-b53e-ea709807a713 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:46.095280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ee265-07ae-4a02-9f22-ad7248dd10ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:46.096148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ee265-07ae-4a02-9f22-ad7248dd10ce admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:46.096437 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ee265-07ae-4a02-9f22-ad7248dd10ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:46.096741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ee265-07ae-4a02-9f22-ad7248dd10ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:46.098592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ee265-07ae-4a02-9f22-ad7248dd10ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:46.099635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ee265-07ae-4a02-9f22-ad7248dd10ce admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:46.151494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54349646-5fd6-4578-914f-4d2cc673af5b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:46.166528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 897/1799] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:46 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:46.166987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54349646-5fd6-4578-914f-4d2cc673af5b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:46.326806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:46.366506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d8e28-c59d-49dd-8a02-8d11565fc84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:46.367537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d8e28-c59d-49dd-8a02-8d11565fc84b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:46.368042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d8e28-c59d-49dd-8a02-8d11565fc84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:46.368661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d8e28-c59d-49dd-8a02-8d11565fc84b admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:46.373878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d8e28-c59d-49dd-8a02-8d11565fc84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:46.379691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d8e28-c59d-49dd-8a02-8d11565fc84b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:46.417126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 903/1800] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:46 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:46.421019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ee265-07ae-4a02-9f22-ad7248dd10ce admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:46.436035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ee265-07ae-4a02-9f22-ad7248dd10ce admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:46.452992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c31fa-aed9-49b1-974d-67df6d79d627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:46.455460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c31fa-aed9-49b1-974d-67df6d79d627 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:46.455758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c31fa-aed9-49b1-974d-67df6d79d627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:46.456041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c31fa-aed9-49b1-974d-67df6d79d627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:46.456786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c31fa-aed9-49b1-974d-67df6d79d627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:46.468302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c31fa-aed9-49b1-974d-67df6d79d627 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:46.494418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d8e28-c59d-49dd-8a02-8d11565fc84b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:46.522571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 898/1801] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:46.522927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d8e28-c59d-49dd-8a02-8d11565fc84b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:46.541908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc999a-2cfe-4583-85f2-eac530c0c658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:46.542734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc999a-2cfe-4583-85f2-eac530c0c658 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:46.543966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc999a-2cfe-4583-85f2-eac530c0c658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:46.544399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc999a-2cfe-4583-85f2-eac530c0c658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:46.545750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc999a-2cfe-4583-85f2-eac530c0c658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:46.549448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc999a-2cfe-4583-85f2-eac530c0c658 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:46.580020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c31fa-aed9-49b1-974d-67df6d79d627 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:46.581416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 904/1802] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:46.604534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c31fa-aed9-49b1-974d-67df6d79d627 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:46.618220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d829f-66d1-471c-aa12-4ce1b3aee21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:46.619474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d829f-66d1-471c-aa12-4ce1b3aee21c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:46.620202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d829f-66d1-471c-aa12-4ce1b3aee21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:46.621384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d829f-66d1-471c-aa12-4ce1b3aee21c admin admin] PATH_INFO: `/v3/projects/f6e0d23905f74636911f2055ff7ecb92/users/5f09069e80534777a3a9e15ac2924b81/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:46.878820 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc999a-2cfe-4583-85f2-eac530c0c658 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:46.881490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b0dc999a-2cfe-4583-85f2-eac530c0c658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:46.896661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d829f-66d1-471c-aa12-4ce1b3aee21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e0d23905f74636911f2055ff7ecb92, user_id=5f09069e80534777a3a9e15ac2924b81, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:46.908137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d829f-66d1-471c-aa12-4ce1b3aee21c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:46.928242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:46.966220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 899/1803] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:46 2022] PUT /identity/v3/projects/f6e0d23905f74636911f2055ff7ecb92/users/5f09069e80534777a3a9e15ac2924b81/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:46.967342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d829f-66d1-471c-aa12-4ce1b3aee21c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:46.981242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-701d829f-66d1-471c-aa12-4ce1b3aee21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:47.075767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc999a-2cfe-4583-85f2-eac530c0c658 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:47.146690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d829f-66d1-471c-aa12-4ce1b3aee21c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:47.152218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:47.153134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:47.153404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:47.153689 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] PATH_INFO: `/v3/users/95c8d51c628444238bedd8931538cc54` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:47.155150 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c8d51c628444238bedd8931538cc54)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:47.157142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:47.212281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556af5e-77d3-44bd-88b9-3cf87b89cfe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:47.212919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556af5e-77d3-44bd-88b9-3cf87b89cfe4 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:47.213282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556af5e-77d3-44bd-88b9-3cf87b89cfe4 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:47.213585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556af5e-77d3-44bd-88b9-3cf87b89cfe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:47.215781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f556af5e-77d3-44bd-88b9-3cf87b89cfe4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:47.240372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c8d51c628444238bedd8931538cc54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:47.241115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c8d51c628444238bedd8931538cc54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:47.255179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c8d51c628444238bedd8931538cc54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:47.260009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c8d51c628444238bedd8931538cc54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:47.278278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556af5e-77d3-44bd-88b9-3cf87b89cfe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:47.279841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556af5e-77d3-44bd-88b9-3cf87b89cfe4 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:47.287927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-37542a58-f16a-4da4-8eda-bca28f113977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c8d51c628444238bedd8931538cc54'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:47.298558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 905/1804] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:47.300296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a8e8c-8337-4cfe-8511-9819eaff6818 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:47.300599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a8e8c-8337-4cfe-8511-9819eaff6818 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:47.300994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a8e8c-8337-4cfe-8511-9819eaff6818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:47.318033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 900/1805] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:46 2022] DELETE /identity/v3/users/95c8d51c628444238bedd8931538cc54 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:47.323890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0a584-9bca-4943-b72e-68b191cd7ef0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:47.324245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0a584-9bca-4943-b72e-68b191cd7ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:47.325361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0a584-9bca-4943-b72e-68b191cd7ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:47.384391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-41a0a584-9bca-4943-b72e-68b191cd7ef0 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:47.386067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-41a0a584-9bca-4943-b72e-68b191cd7ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:47.387679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c39a8e8c-8337-4cfe-8511-9819eaff6818 None None] MFA Rules not processed for user `5f09069e80534777a3a9e15ac2924b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:47.388879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c39a8e8c-8337-4cfe-8511-9819eaff6818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:47.504758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:47.570829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 906/1806] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:47.571696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a8e8c-8337-4cfe-8511-9819eaff6818 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:47.584499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c39a8e8c-8337-4cfe-8511-9819eaff6818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:47.659278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 901/1807] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:47.778967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a8e8c-8337-4cfe-8511-9819eaff6818 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:47.788642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7785bbd-8bc0-4b80-b468-7bd3ed420198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:47.791719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7785bbd-8bc0-4b80-b468-7bd3ed420198 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:47.792759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7785bbd-8bc0-4b80-b468-7bd3ed420198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:47.793292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7785bbd-8bc0-4b80-b468-7bd3ed420198 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:47.794262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7785bbd-8bc0-4b80-b468-7bd3ed420198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:47.797018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7785bbd-8bc0-4b80-b468-7bd3ed420198 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:47.824336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0a584-9bca-4943-b72e-68b191cd7ef0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:47.829065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-41a0a584-9bca-4943-b72e-68b191cd7ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:47.872237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 907/1808] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:47 2022] POST /identity/v3/projects => generated 385 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:47.876064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7785bbd-8bc0-4b80-b468-7bd3ed420198 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:47.898562 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:48.006542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7785bbd-8bc0-4b80-b468-7bd3ed420198 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:48.013424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e22cf6-6334-4860-a0b4-62efbb855f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:48.014262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e22cf6-6334-4860-a0b4-62efbb855f97 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.014643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e22cf6-6334-4860-a0b4-62efbb855f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.015017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e22cf6-6334-4860-a0b4-62efbb855f97 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.015862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e22cf6-6334-4860-a0b4-62efbb855f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:48.018148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e22cf6-6334-4860-a0b4-62efbb855f97 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:48.039237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0a584-9bca-4943-b72e-68b191cd7ef0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:48.049713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 908/1809] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:47 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:48.053454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e22cf6-6334-4860-a0b4-62efbb855f97 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:48.069043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e22cf6-6334-4860-a0b4-62efbb855f97 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:48.074942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d81bb6-0129-40be-9a11-e340970036a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:48.075738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d81bb6-0129-40be-9a11-e340970036a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.076094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d81bb6-0129-40be-9a11-e340970036a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.076696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d81bb6-0129-40be-9a11-e340970036a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.077339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d81bb6-0129-40be-9a11-e340970036a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:48.079775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d81bb6-0129-40be-9a11-e340970036a9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:48.096189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 909/1810] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:34:48.100710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d81bb6-0129-40be-9a11-e340970036a9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:48.120784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d81bb6-0129-40be-9a11-e340970036a9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:48.136674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20950d0a-aa5c-426b-afc6-704fcb1f3995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:48.137670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20950d0a-aa5c-426b-afc6-704fcb1f3995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.138017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20950d0a-aa5c-426b-afc6-704fcb1f3995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.138394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20950d0a-aa5c-426b-afc6-704fcb1f3995 admin admin] PATH_INFO: `/v3/projects/3018c3338a784078851cefb3276e60ab/users/82cf621112d34606ae4d45da525f3dc4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.164075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20950d0a-aa5c-426b-afc6-704fcb1f3995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3018c3338a784078851cefb3276e60ab, user_id=82cf621112d34606ae4d45da525f3dc4, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:48.167852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20950d0a-aa5c-426b-afc6-704fcb1f3995 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:48.187014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ff705-3b9f-4431-a635-e78f77e00158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:48.191395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 910/1811] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:48 2022] PUT /identity/v3/projects/3018c3338a784078851cefb3276e60ab/users/82cf621112d34606ae4d45da525f3dc4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:48.192918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ff705-3b9f-4431-a635-e78f77e00158 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.193275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ff705-3b9f-4431-a635-e78f77e00158 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.193565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ff705-3b9f-4431-a635-e78f77e00158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.194107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20950d0a-aa5c-426b-afc6-704fcb1f3995 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:48.221611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:48.232952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ff705-3b9f-4431-a635-e78f77e00158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:48.238887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ff705-3b9f-4431-a635-e78f77e00158 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:48.268830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 902/1812] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:48.300863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20950d0a-aa5c-426b-afc6-704fcb1f3995 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:48.307120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afc55b-b14f-4e18-9356-283caa51212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:48.307985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afc55b-b14f-4e18-9356-283caa51212c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.309007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afc55b-b14f-4e18-9356-283caa51212c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.309007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afc55b-b14f-4e18-9356-283caa51212c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.309671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afc55b-b14f-4e18-9356-283caa51212c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:48.313862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afc55b-b14f-4e18-9356-283caa51212c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:48.332947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 911/1813] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:48.334309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ff705-3b9f-4431-a635-e78f77e00158 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:48.345856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ff705-3b9f-4431-a635-e78f77e00158 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:48.351118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deb360-ec81-41da-bd92-0babb5b4faa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:48.352220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deb360-ec81-41da-bd92-0babb5b4faa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.352498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deb360-ec81-41da-bd92-0babb5b4faa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.352770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deb360-ec81-41da-bd92-0babb5b4faa5 admin admin] PATH_INFO: `/v3/projects/3018c3338a784078851cefb3276e60ab/users/82cf621112d34606ae4d45da525f3dc4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.354796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9deb360-ec81-41da-bd92-0babb5b4faa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3018c3338a784078851cefb3276e60ab, user_id=82cf621112d34606ae4d45da525f3dc4, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:48.357480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9deb360-ec81-41da-bd92-0babb5b4faa5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:48.369720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 903/1814] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:48 2022] PUT /identity/v3/projects/3018c3338a784078851cefb3276e60ab/users/82cf621112d34606ae4d45da525f3dc4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:48.379898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d2b2e-7ddc-41a1-a127-7f8cf521fa2a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.380435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d2b2e-7ddc-41a1-a127-7f8cf521fa2a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.380802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d2b2e-7ddc-41a1-a127-7f8cf521fa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.419061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c03d2b2e-7ddc-41a1-a127-7f8cf521fa2a None None] MFA Rules not processed for user `82cf621112d34606ae4d45da525f3dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:48.420952 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c03d2b2e-7ddc-41a1-a127-7f8cf521fa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:48.540221 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 912/1815] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:48.549867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deb360-ec81-41da-bd92-0babb5b4faa5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:48.551851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d9deb360-ec81-41da-bd92-0babb5b4faa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:48.645922 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deb360-ec81-41da-bd92-0babb5b4faa5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:48.650194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170c8ab-b98e-457e-a5b7-80bb3cc5cebf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:48.651064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170c8ab-b98e-457e-a5b7-80bb3cc5cebf service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.651811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170c8ab-b98e-457e-a5b7-80bb3cc5cebf service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.652250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170c8ab-b98e-457e-a5b7-80bb3cc5cebf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.677944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170c8ab-b98e-457e-a5b7-80bb3cc5cebf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:48.679692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170c8ab-b98e-457e-a5b7-80bb3cc5cebf service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:48.706357 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 904/1816] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:48.747225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d2b2e-7ddc-41a1-a127-7f8cf521fa2a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:48.830004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d2b2e-7ddc-41a1-a127-7f8cf521fa2a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:48.836753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:48.838435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.838435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.838435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] PATH_INFO: `/v3/users/5f09069e80534777a3a9e15ac2924b81` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.839468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f09069e80534777a3a9e15ac2924b81)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:48.841712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:48.910088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:48.914366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f09069e80534777a3a9e15ac2924b81'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:48.914993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f09069e80534777a3a9e15ac2924b81'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:48.932697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f09069e80534777a3a9e15ac2924b81'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:48.940488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170c8ab-b98e-457e-a5b7-80bb3cc5cebf service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:48.942188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e170c8ab-b98e-457e-a5b7-80bb3cc5cebf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:48.956442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f09069e80534777a3a9e15ac2924b81'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:48.957405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-86e22496-084d-42eb-b829-f27fbf3bae31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f09069e80534777a3a9e15ac2924b81'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:48.962234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170c8ab-b98e-457e-a5b7-80bb3cc5cebf service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:48.967612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:48.968776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.969246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.969694 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] PATH_INFO: `/v3/projects/441248be0f8f4f79b1411411e24cb636` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:48.971789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=441248be0f8f4f79b1411411e24cb636)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:48.973926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:48.977502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 913/1817] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:48 2022] DELETE /identity/v3/users/5f09069e80534777a3a9e15ac2924b81 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:48.982090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95668757-dc77-4002-983b-945c408f4cee None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:48.982609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95668757-dc77-4002-983b-945c408f4cee None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:48.982945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95668757-dc77-4002-983b-945c408f4cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:49.008631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-95668757-dc77-4002-983b-945c408f4cee None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:49.010065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-95668757-dc77-4002-983b-945c408f4cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:49.047755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441248be0f8f4f79b1411411e24cb636', 'request_id': 'req-be59441b-4464-4ebb-aaa6-9ffd47830d84'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.062116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441248be0f8f4f79b1411411e24cb636', 'request_id': 'req-be59441b-4464-4ebb-aaa6-9ffd47830d84'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.077883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441248be0f8f4f79b1411411e24cb636', 'request_id': 'req-be59441b-4464-4ebb-aaa6-9ffd47830d84'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.079751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:49.092313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] The token cache is being invalidate because project 441248be0f8f4f79b1411411e24cb636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:49.093898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.096280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.098719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 905/1818] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:48 2022] DELETE /identity/v3/projects/441248be0f8f4f79b1411411e24cb636 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:49.099971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 914/1819] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:49.116076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:49.124652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:49.137194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95668757-dc77-4002-983b-945c408f4cee None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:49.138775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-95668757-dc77-4002-983b-945c408f4cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:49.256674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59441b-4464-4ebb-aaa6-9ffd47830d84 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:49.263250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:49.264306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:49.264685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:49.265022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] PATH_INFO: `/v3/users/0db908f3bafc4d93b4b8f0493c6a1fbf` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:49.266553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db908f3bafc4d93b4b8f0493c6a1fbf)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:49.268956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:49.275185 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95668757-dc77-4002-983b-945c408f4cee None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:49.364344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db908f3bafc4d93b4b8f0493c6a1fbf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.365134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c08d84-ffdc-4f36-9fa9-e2ef7542d133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:49.365309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db908f3bafc4d93b4b8f0493c6a1fbf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.366288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c08d84-ffdc-4f36-9fa9-e2ef7542d133 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:49.368881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c08d84-ffdc-4f36-9fa9-e2ef7542d133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:49.369378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c08d84-ffdc-4f36-9fa9-e2ef7542d133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:49.372768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-92c08d84-ffdc-4f36-9fa9-e2ef7542d133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:49.380669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db908f3bafc4d93b4b8f0493c6a1fbf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.381613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db908f3bafc4d93b4b8f0493c6a1fbf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.399766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f59dbfa6-533a-47e2-93d9-02b0a9c0922a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db908f3bafc4d93b4b8f0493c6a1fbf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.418052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 906/1820] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:49 2022] DELETE /identity/v3/users/0db908f3bafc4d93b4b8f0493c6a1fbf => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:49.419681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c08d84-ffdc-4f36-9fa9-e2ef7542d133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:49.419814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaa530-2eb0-4f2b-a1f0-8f35d434ed7b None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:49.419982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaa530-2eb0-4f2b-a1f0-8f35d434ed7b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:49.420573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaa530-2eb0-4f2b-a1f0-8f35d434ed7b None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:49.421954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 907/1821] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:34:49.422931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c08d84-ffdc-4f36-9fa9-e2ef7542d133 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:49.441208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaa530-2eb0-4f2b-a1f0-8f35d434ed7b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:49.443249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bfcaa530-2eb0-4f2b-a1f0-8f35d434ed7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:49.453424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 915/1822] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:49.458874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71255b-8e7b-4c50-9499-81baf911343a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:49.459322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71255b-8e7b-4c50-9499-81baf911343a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:49.459693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71255b-8e7b-4c50-9499-81baf911343a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:49.493292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1b71255b-8e7b-4c50-9499-81baf911343a None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:49.495514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1b71255b-8e7b-4c50-9499-81baf911343a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:49.531466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:49.584747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaa530-2eb0-4f2b-a1f0-8f35d434ed7b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:49.597609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:49.599365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:49.605098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:49.605954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] PATH_INFO: `/v3/projects/8a8989289a0e4cfb9c60f0e07ef3b09d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:49.608849 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a8989289a0e4cfb9c60f0e07ef3b09d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:49.611861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:49.624919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:49.639480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 916/1823] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:34:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:49.655527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71255b-8e7b-4c50-9499-81baf911343a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:49.658202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1b71255b-8e7b-4c50-9499-81baf911343a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:49.674259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71255b-8e7b-4c50-9499-81baf911343a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:49.679470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a8989289a0e4cfb9c60f0e07ef3b09d', 'request_id': 'req-09f0eda4-ac13-4d32-af1c-dbd83811318c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.688937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a8989289a0e4cfb9c60f0e07ef3b09d', 'request_id': 'req-09f0eda4-ac13-4d32-af1c-dbd83811318c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.698524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a8989289a0e4cfb9c60f0e07ef3b09d', 'request_id': 'req-09f0eda4-ac13-4d32-af1c-dbd83811318c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.715676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] The token cache is being invalidate because project 8a8989289a0e4cfb9c60f0e07ef3b09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:49.716799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.717910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.723867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 908/1824] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:49 2022] DELETE /identity/v3/projects/8a8989289a0e4cfb9c60f0e07ef3b09d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:49.725885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:49.727854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:49.805807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:49.806055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:49.806304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:49.806580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:49.810429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f0eda4-ac13-4d32-af1c-dbd83811318c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:49.814156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:49.816053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:49.817457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:49.817839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:49.818317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] PATH_INFO: `/v3/users/f31c40f46aa246448a5d4e4bbe66bd08` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:49.819917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f31c40f46aa246448a5d4e4bbe66bd08)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:49.822218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:49.827713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:49.829394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:49.855681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 917/1825] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:34:49.920560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f31c40f46aa246448a5d4e4bbe66bd08'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.921338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f31c40f46aa246448a5d4e4bbe66bd08'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.930823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f31c40f46aa246448a5d4e4bbe66bd08'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.931578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f31c40f46aa246448a5d4e4bbe66bd08'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.944331 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f31c40f46aa246448a5d4e4bbe66bd08'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:49.957110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 909/1826] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:49 2022] DELETE /identity/v3/users/f31c40f46aa246448a5d4e4bbe66bd08 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:50.112709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:50.116087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:50.132520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3fe49-04ca-4434-92a3-469b20e3ccb2 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:50.149586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:50.150905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:50.181751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:50.232831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:50.234051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.234316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.234660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] PATH_INFO: `/v3/projects/f6e0d23905f74636911f2055ff7ecb92` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.239937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e0d23905f74636911f2055ff7ecb92)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:50.241989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:50.242631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c35b77-3243-47ea-8a23-f1a102f73064 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:50.247624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:50.248500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.248795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.249059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] PATH_INFO: `/v3/projects/9a3c2ce52b1849979f0748e6597d195a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.250251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a3c2ce52b1849979f0748e6597d195a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:50.252220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:50.287092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:50.296043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e0d23905f74636911f2055ff7ecb92', 'request_id': 'req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.303745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a3c2ce52b1849979f0748e6597d195a', 'request_id': 'req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.309293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e0d23905f74636911f2055ff7ecb92', 'request_id': 'req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.313786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a3c2ce52b1849979f0748e6597d195a', 'request_id': 'req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.320143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e0d23905f74636911f2055ff7ecb92', 'request_id': 'req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.327632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a3c2ce52b1849979f0748e6597d195a', 'request_id': 'req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.341343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] The token cache is being invalidate because project 9a3c2ce52b1849979f0748e6597d195a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:50.341844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.344328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.348949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 910/1827] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:50 2022] DELETE /identity/v3/projects/9a3c2ce52b1849979f0748e6597d195a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:50.348949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:50.348949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:50.350216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] The token cache is being invalidate because project f6e0d23905f74636911f2055ff7ecb92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:50.350513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.351526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fdfbf88-a1a4-4bab-9e3e-5045cd29a12e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.353620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 918/1828] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:50 2022] DELETE /identity/v3/projects/f6e0d23905f74636911f2055ff7ecb92 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:50.357649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08e96-cfa2-4ee6-b12f-5bff724ab7cd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.357903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08e96-cfa2-4ee6-b12f-5bff724ab7cd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.358190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08e96-cfa2-4ee6-b12f-5bff724ab7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.378290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1fc08e96-cfa2-4ee6-b12f-5bff724ab7cd None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:50.379613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1fc08e96-cfa2-4ee6-b12f-5bff724ab7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:50.430868 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea5d54-d12e-41a2-afeb-e24a1f2c79d8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:50.437185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:50.438215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.438690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.439031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] PATH_INFO: `/v3/projects/eef87dfe92b2434695779feddfafe39c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.440485 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef87dfe92b2434695779feddfafe39c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:50.443766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:50.509030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef87dfe92b2434695779feddfafe39c', 'request_id': 'req-36dfda9a-89d2-45a8-88ae-152b56d6b422'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.511390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:50.521799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef87dfe92b2434695779feddfafe39c', 'request_id': 'req-36dfda9a-89d2-45a8-88ae-152b56d6b422'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.530135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef87dfe92b2434695779feddfafe39c', 'request_id': 'req-36dfda9a-89d2-45a8-88ae-152b56d6b422'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.542810 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] The token cache is being invalidate because project eef87dfe92b2434695779feddfafe39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:50.543105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.543854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.546066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 911/1829] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:50 2022] DELETE /identity/v3/projects/eef87dfe92b2434695779feddfafe39c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:50.546617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:50.548044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:50.580067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 919/1830] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:50.581479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e5d3-0ba7-4eba-a148-1a1bef71d864 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.581768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e5d3-0ba7-4eba-a148-1a1bef71d864 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.582060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e5d3-0ba7-4eba-a148-1a1bef71d864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.607173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8b52e5d3-0ba7-4eba-a148-1a1bef71d864 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:50.608462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8b52e5d3-0ba7-4eba-a148-1a1bef71d864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:50.629568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfda9a-89d2-45a8-88ae-152b56d6b422 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:50.634873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:50.636024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.636155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.636341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] PATH_INFO: `/v3/users/82cf621112d34606ae4d45da525f3dc4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.637636 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] RBAC: Authorizing `identity:delete_user(user_id=82cf621112d34606ae4d45da525f3dc4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:50.639726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:50.676336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 920/1831] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:50.677687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b262-fbd3-45cd-a8c0-2be83c55779d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.677976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b262-fbd3-45cd-a8c0-2be83c55779d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.678250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b262-fbd3-45cd-a8c0-2be83c55779d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.707608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82cf621112d34606ae4d45da525f3dc4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.708304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82cf621112d34606ae4d45da525f3dc4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.719808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-adc6b262-fbd3-45cd-a8c0-2be83c55779d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:50.719808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-adc6b262-fbd3-45cd-a8c0-2be83c55779d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:50.719995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82cf621112d34606ae4d45da525f3dc4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.720839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82cf621112d34606ae4d45da525f3dc4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.730735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 921/1832] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:50.732753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0c32e-548c-4bf0-95bc-0f166d736821 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.733128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0c32e-548c-4bf0-95bc-0f166d736821 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.733807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0c32e-548c-4bf0-95bc-0f166d736821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.738262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82cf621112d34606ae4d45da525f3dc4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:50.758919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5be0c32e-548c-4bf0-95bc-0f166d736821 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:50.762022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 912/1833] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:50 2022] DELETE /identity/v3/users/82cf621112d34606ae4d45da525f3dc4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:50.762175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5be0c32e-548c-4bf0-95bc-0f166d736821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:50.762268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:50.863581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 922/1834] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:50.868019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0c32e-548c-4bf0-95bc-0f166d736821 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:50.873758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:50.880597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0c32e-548c-4bf0-95bc-0f166d736821 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:50.887889 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb12bc-c8c2-404c-9374-1b0e13537021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:50.888809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb12bc-c8c2-404c-9374-1b0e13537021 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.889449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb12bc-c8c2-404c-9374-1b0e13537021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.889811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeb12bc-c8c2-404c-9374-1b0e13537021 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.891061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb12bc-c8c2-404c-9374-1b0e13537021 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:50.892050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb12bc-c8c2-404c-9374-1b0e13537021 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:50.894526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeb12bc-c8c2-404c-9374-1b0e13537021 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:50.898098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0d5f0-8efd-49bb-8393-ea5c4364432e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:50.899311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:50.904588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7be89a-880c-4fc8-b5c8-1d3c0b806d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:50.905685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7be89a-880c-4fc8-b5c8-1d3c0b806d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.906508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7be89a-880c-4fc8-b5c8-1d3c0b806d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.906508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7be89a-880c-4fc8-b5c8-1d3c0b806d74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.910760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 923/1835] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:34:50.912302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7be89a-880c-4fc8-b5c8-1d3c0b806d74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:50.912302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7be89a-880c-4fc8-b5c8-1d3c0b806d74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:50.912302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7be89a-880c-4fc8-b5c8-1d3c0b806d74 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:50.912633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb12bc-c8c2-404c-9374-1b0e13537021 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:50.930206 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 913/1836] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:50.932361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeb12bc-c8c2-404c-9374-1b0e13537021 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:50.936171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7be89a-880c-4fc8-b5c8-1d3c0b806d74 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:50.938927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9e86a-cb48-42f9-a685-59f05568152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:50.939645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9e86a-cb48-42f9-a685-59f05568152f admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.939924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9e86a-cb48-42f9-a685-59f05568152f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.940196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9e86a-cb48-42f9-a685-59f05568152f admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.940888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9e86a-cb48-42f9-a685-59f05568152f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:50.942810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9e86a-cb48-42f9-a685-59f05568152f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:50.967529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7be89a-880c-4fc8-b5c8-1d3c0b806d74 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:50.972640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc1e45-012a-4ad8-9cf2-90df1469bba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:50.973304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc1e45-012a-4ad8-9cf2-90df1469bba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:50.973585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc1e45-012a-4ad8-9cf2-90df1469bba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:50.973864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc1e45-012a-4ad8-9cf2-90df1469bba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:50.974554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc1e45-012a-4ad8-9cf2-90df1469bba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:50.976750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecc1e45-012a-4ad8-9cf2-90df1469bba9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:50.976929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 924/1837] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:50 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:50.980794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9e86a-cb48-42f9-a685-59f05568152f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.011313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 914/1838] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:50 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:51.012372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc1e45-012a-4ad8-9cf2-90df1469bba9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.055871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9e86a-cb48-42f9-a685-59f05568152f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.076930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc1e45-012a-4ad8-9cf2-90df1469bba9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.081929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5c510-5d97-4015-80df-0e93e5f527ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.082605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5c510-5d97-4015-80df-0e93e5f527ae admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.082878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5c510-5d97-4015-80df-0e93e5f527ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.083158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5c510-5d97-4015-80df-0e93e5f527ae admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.083874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5c510-5d97-4015-80df-0e93e5f527ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.086060 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5c510-5d97-4015-80df-0e93e5f527ae admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.108801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 915/1839] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:51.112489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5c510-5d97-4015-80df-0e93e5f527ae admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.122882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5c510-5d97-4015-80df-0e93e5f527ae admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.133139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38f2b0-82ee-464a-b0d4-f94af6220745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.134187 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38f2b0-82ee-464a-b0d4-f94af6220745 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.134293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38f2b0-82ee-464a-b0d4-f94af6220745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.134636 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38f2b0-82ee-464a-b0d4-f94af6220745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.135466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38f2b0-82ee-464a-b0d4-f94af6220745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.137177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38f2b0-82ee-464a-b0d4-f94af6220745 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.137992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719dad0-2083-4dde-9b43-5d14ff076522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.138781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719dad0-2083-4dde-9b43-5d14ff076522 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.139070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719dad0-2083-4dde-9b43-5d14ff076522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.139476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719dad0-2083-4dde-9b43-5d14ff076522 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.140274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719dad0-2083-4dde-9b43-5d14ff076522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.142247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719dad0-2083-4dde-9b43-5d14ff076522 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.149712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 916/1840] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:34:51.153450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38f2b0-82ee-464a-b0d4-f94af6220745 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.168404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38f2b0-82ee-464a-b0d4-f94af6220745 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.172542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 925/1841] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:50 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:51.173494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719dad0-2083-4dde-9b43-5d14ff076522 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.179555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24008c3-5ca0-40bd-ab64-7f946d01d6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.180937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24008c3-5ca0-40bd-ab64-7f946d01d6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.181236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24008c3-5ca0-40bd-ab64-7f946d01d6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.184158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24008c3-5ca0-40bd-ab64-7f946d01d6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.184158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24008c3-5ca0-40bd-ab64-7f946d01d6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.184158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24008c3-5ca0-40bd-ab64-7f946d01d6f9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.187783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719dad0-2083-4dde-9b43-5d14ff076522 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.192917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fc9d7-2ad4-4f6a-b2bc-7fbfa26ce7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.193666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fc9d7-2ad4-4f6a-b2bc-7fbfa26ce7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.194018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fc9d7-2ad4-4f6a-b2bc-7fbfa26ce7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.194361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fc9d7-2ad4-4f6a-b2bc-7fbfa26ce7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.195254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fc9d7-2ad4-4f6a-b2bc-7fbfa26ce7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.195709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 917/1842] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:51.196697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24008c3-5ca0-40bd-ab64-7f946d01d6f9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.197663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fc9d7-2ad4-4f6a-b2bc-7fbfa26ce7b3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.199016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a24008c3-5ca0-40bd-ab64-7f946d01d6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:51.209426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 926/1843] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:51.210457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fc9d7-2ad4-4f6a-b2bc-7fbfa26ce7b3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.215114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24008c3-5ca0-40bd-ab64-7f946d01d6f9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.221604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.222679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.223082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.223411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fc9d7-2ad4-4f6a-b2bc-7fbfa26ce7b3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.223666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] PATH_INFO: `/v3/projects/3018c3338a784078851cefb3276e60ab` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.224964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3018c3338a784078851cefb3276e60ab)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.227046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.229027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aee4db-13b0-41d2-806d-956c4811bb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.231697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aee4db-13b0-41d2-806d-956c4811bb63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.231955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aee4db-13b0-41d2-806d-956c4811bb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.232238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aee4db-13b0-41d2-806d-956c4811bb63 admin admin] PATH_INFO: `/v3/projects/5daa8d3eaf2540918f6e0d3fbe987135/users/fcf60fa06c8a439fba19b3165396eb64/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.248765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aee4db-13b0-41d2-806d-956c4811bb63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5daa8d3eaf2540918f6e0d3fbe987135, user_id=fcf60fa06c8a439fba19b3165396eb64, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.251663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aee4db-13b0-41d2-806d-956c4811bb63 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.269732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 927/1844] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:51 2022] PUT /identity/v3/projects/5daa8d3eaf2540918f6e0d3fbe987135/users/fcf60fa06c8a439fba19b3165396eb64/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:51.270685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aee4db-13b0-41d2-806d-956c4811bb63 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.279631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3018c3338a784078851cefb3276e60ab', 'request_id': 'req-f9845695-9410-4993-a994-c0d999ed2c26'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:51.289453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3018c3338a784078851cefb3276e60ab', 'request_id': 'req-f9845695-9410-4993-a994-c0d999ed2c26'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:51.296609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3018c3338a784078851cefb3276e60ab', 'request_id': 'req-f9845695-9410-4993-a994-c0d999ed2c26'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:51.305742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:51.307678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] The token cache is being invalidate because project 3018c3338a784078851cefb3276e60ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:51.308015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:51.308652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f9845695-9410-4993-a994-c0d999ed2c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:51.310417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 918/1845] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:51 2022] DELETE /identity/v3/projects/3018c3338a784078851cefb3276e60ab => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:51.311548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e68fa-cf2f-4671-9031-c34b5d789e31 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.311833 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e68fa-cf2f-4671-9031-c34b5d789e31 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.312109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e68fa-cf2f-4671-9031-c34b5d789e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.338552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-246e68fa-cf2f-4671-9031-c34b5d789e31 None None] MFA Rules not processed for user `fcf60fa06c8a439fba19b3165396eb64`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:51.339700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-246e68fa-cf2f-4671-9031-c34b5d789e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:51.340005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aee4db-13b0-41d2-806d-956c4811bb63 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.348503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:51.398725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91d98a-8202-4a24-bb1d-f773f573dfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.399550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91d98a-8202-4a24-bb1d-f773f573dfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.399831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91d98a-8202-4a24-bb1d-f773f573dfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.400103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91d98a-8202-4a24-bb1d-f773f573dfc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.400824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91d98a-8202-4a24-bb1d-f773f573dfc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.402427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91d98a-8202-4a24-bb1d-f773f573dfc1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.407466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 919/1846] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:51.408745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022581dc-4dc2-4008-a686-f8c6ba3cb150 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.409053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022581dc-4dc2-4008-a686-f8c6ba3cb150 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.409317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022581dc-4dc2-4008-a686-f8c6ba3cb150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.415694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 928/1847] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:51.415694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91d98a-8202-4a24-bb1d-f773f573dfc1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.415694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1e91d98a-8202-4a24-bb1d-f773f573dfc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:51.427778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91d98a-8202-4a24-bb1d-f773f573dfc1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.428551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-022581dc-4dc2-4008-a686-f8c6ba3cb150 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:51.429937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-022581dc-4dc2-4008-a686-f8c6ba3cb150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:51.432971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78b7b1-d72b-4a88-94e2-72509ca91eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.433613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b7b1-d72b-4a88-94e2-72509ca91eea admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.434266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b7b1-d72b-4a88-94e2-72509ca91eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.434266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b7b1-d72b-4a88-94e2-72509ca91eea admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.434819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78b7b1-d72b-4a88-94e2-72509ca91eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.436792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78b7b1-d72b-4a88-94e2-72509ca91eea admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.438225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 920/1848] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:51.440722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022581dc-4dc2-4008-a686-f8c6ba3cb150 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.442896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-022581dc-4dc2-4008-a686-f8c6ba3cb150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:51.458688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022581dc-4dc2-4008-a686-f8c6ba3cb150 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.463546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5954d-1d20-4d5a-84f2-68e22bfcc27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.464435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5954d-1d20-4d5a-84f2-68e22bfcc27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.464710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5954d-1d20-4d5a-84f2-68e22bfcc27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.464984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5954d-1d20-4d5a-84f2-68e22bfcc27f admin admin] PATH_INFO: `/v3/projects/58e6bc94f4d44deeb1341c8a74e5c5f4/users/fe3c90fd557340c498c575824fdc7cdd/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.465933 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 929/1849] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:51.469568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828435a3-3697-4a33-be17-92cd958a8002 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.469866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828435a3-3697-4a33-be17-92cd958a8002 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.470147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828435a3-3697-4a33-be17-92cd958a8002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.479722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5954d-1d20-4d5a-84f2-68e22bfcc27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=58e6bc94f4d44deeb1341c8a74e5c5f4, user_id=fe3c90fd557340c498c575824fdc7cdd, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.482372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5954d-1d20-4d5a-84f2-68e22bfcc27f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.487952 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-828435a3-3697-4a33-be17-92cd958a8002 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:51.489204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-828435a3-3697-4a33-be17-92cd958a8002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:51.496045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 921/1850] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:51 2022] PUT /identity/v3/projects/58e6bc94f4d44deeb1341c8a74e5c5f4/users/fe3c90fd557340c498c575824fdc7cdd/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:51.504733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5954d-1d20-4d5a-84f2-68e22bfcc27f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.549941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 930/1851] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:51.551348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0385387-b3a2-4d8b-b4e5-327881773a0c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.551641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0385387-b3a2-4d8b-b4e5-327881773a0c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.551921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0385387-b3a2-4d8b-b4e5-327881773a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.560114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5954d-1d20-4d5a-84f2-68e22bfcc27f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.564780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb4a17-0ca1-494f-838f-3adacd5e58d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.565450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb4a17-0ca1-494f-838f-3adacd5e58d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.565739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb4a17-0ca1-494f-838f-3adacd5e58d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.566013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb4a17-0ca1-494f-838f-3adacd5e58d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.566721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb4a17-0ca1-494f-838f-3adacd5e58d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.568686 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb4a17-0ca1-494f-838f-3adacd5e58d7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.579946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d0385387-b3a2-4d8b-b4e5-327881773a0c None None] MFA Rules not processed for user `fe3c90fd557340c498c575824fdc7cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:51.581377 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d0385387-b3a2-4d8b-b4e5-327881773a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:51.590631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 922/1852] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:51.591427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb4a17-0ca1-494f-838f-3adacd5e58d7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.600680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb4a17-0ca1-494f-838f-3adacd5e58d7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.605517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c8b94-f59e-4e57-8ff2-79ddb71b4111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.606157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c8b94-f59e-4e57-8ff2-79ddb71b4111 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.606439 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c8b94-f59e-4e57-8ff2-79ddb71b4111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.606711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c8b94-f59e-4e57-8ff2-79ddb71b4111 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.607452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c8b94-f59e-4e57-8ff2-79ddb71b4111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:51.607756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c8b94-f59e-4e57-8ff2-79ddb71b4111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.609339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c8b94-f59e-4e57-8ff2-79ddb71b4111 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.621355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 923/1853] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:51.622155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c8b94-f59e-4e57-8ff2-79ddb71b4111 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.627324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:51.632114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c8b94-f59e-4e57-8ff2-79ddb71b4111 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.636820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cfe79-94ae-41b4-9b02-10b86b39b9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.637710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cfe79-94ae-41b4-9b02-10b86b39b9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.637710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cfe79-94ae-41b4-9b02-10b86b39b9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.637999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7cfe79-94ae-41b4-9b02-10b86b39b9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.638711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cfe79-94ae-41b4-9b02-10b86b39b9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.640324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7cfe79-94ae-41b4-9b02-10b86b39b9a8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.650191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 924/1854] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:51.650957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cfe79-94ae-41b4-9b02-10b86b39b9a8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.656608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 931/1855] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:51.660373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0385387-b3a2-4d8b-b4e5-327881773a0c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.660641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7cfe79-94ae-41b4-9b02-10b86b39b9a8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.666451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94301d48-8e3f-4d13-81f0-1940dc9cf376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.667109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94301d48-8e3f-4d13-81f0-1940dc9cf376 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.667426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94301d48-8e3f-4d13-81f0-1940dc9cf376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.667708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94301d48-8e3f-4d13-81f0-1940dc9cf376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.668404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94301d48-8e3f-4d13-81f0-1940dc9cf376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.670632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94301d48-8e3f-4d13-81f0-1940dc9cf376 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.672523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0385387-b3a2-4d8b-b4e5-327881773a0c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.677829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde397d-f000-4d0a-b61f-42b2d63a7d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.678495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde397d-f000-4d0a-b61f-42b2d63a7d5c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.678789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde397d-f000-4d0a-b61f-42b2d63a7d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.679086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde397d-f000-4d0a-b61f-42b2d63a7d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.680721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde397d-f000-4d0a-b61f-42b2d63a7d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.682359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde397d-f000-4d0a-b61f-42b2d63a7d5c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.693220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 932/1856] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:51.694055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde397d-f000-4d0a-b61f-42b2d63a7d5c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.696688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8fde397d-f000-4d0a-b61f-42b2d63a7d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:51.700012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 925/1857] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:51.700709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94301d48-8e3f-4d13-81f0-1940dc9cf376 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.725227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:51.761089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde397d-f000-4d0a-b61f-42b2d63a7d5c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.775723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94301d48-8e3f-4d13-81f0-1940dc9cf376 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.780642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc462c29-25b0-4697-b577-cd399a41f3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.781583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462c29-25b0-4697-b577-cd399a41f3a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.781871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462c29-25b0-4697-b577-cd399a41f3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.782151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc462c29-25b0-4697-b577-cd399a41f3a1 admin admin] PATH_INFO: `/v3/projects/91cf5f624b2e4584826f1c3d8511bd08/users/d3ba3423897e42cfaf4be024eddda8c5/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.798779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc462c29-25b0-4697-b577-cd399a41f3a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91cf5f624b2e4584826f1c3d8511bd08, user_id=d3ba3423897e42cfaf4be024eddda8c5, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.801684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc462c29-25b0-4697-b577-cd399a41f3a1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.821779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4a535-4366-42c8-b080-f05b13ea665b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.823355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4a535-4366-42c8-b080-f05b13ea665b service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.823703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4a535-4366-42c8-b080-f05b13ea665b service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.823812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 926/1858] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:51 2022] PUT /identity/v3/projects/91cf5f624b2e4584826f1c3d8511bd08/users/d3ba3423897e42cfaf4be024eddda8c5/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:51.824158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4a535-4366-42c8-b080-f05b13ea665b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.824784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc462c29-25b0-4697-b577-cd399a41f3a1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.907639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc462c29-25b0-4697-b577-cd399a41f3a1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.912562 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1bafd-1823-4213-8365-0f31f5053aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.913274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1bafd-1823-4213-8365-0f31f5053aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.913841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1bafd-1823-4213-8365-0f31f5053aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.913841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1bafd-1823-4213-8365-0f31f5053aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.914741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1bafd-1823-4213-8365-0f31f5053aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.916855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1bafd-1823-4213-8365-0f31f5053aa2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.922282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4a535-4366-42c8-b080-f05b13ea665b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.923759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4a535-4366-42c8-b080-f05b13ea665b service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:51.940134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 927/1859] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:51.941511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e5276-5b68-4b9e-a8cc-7ac77fc0a246 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.941876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e5276-5b68-4b9e-a8cc-7ac77fc0a246 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.942206 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e5276-5b68-4b9e-a8cc-7ac77fc0a246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.944010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 933/1860] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:51.944493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4a535-4366-42c8-b080-f05b13ea665b service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:51.965960 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4a535-4366-42c8-b080-f05b13ea665b service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:51.980599 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0d9e5276-5b68-4b9e-a8cc-7ac77fc0a246 None None] MFA Rules not processed for user `d3ba3423897e42cfaf4be024eddda8c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:51.982091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0d9e5276-5b68-4b9e-a8cc-7ac77fc0a246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:51.984142 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c2b57-e9f0-401f-a9d3-abb9d7a18b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:51.984842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c2b57-e9f0-401f-a9d3-abb9d7a18b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:51.985125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c2b57-e9f0-401f-a9d3-abb9d7a18b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:51.985399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c2b57-e9f0-401f-a9d3-abb9d7a18b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:51.986282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c2b57-e9f0-401f-a9d3-abb9d7a18b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:51.989884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c2b57-e9f0-401f-a9d3-abb9d7a18b5d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.013402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 934/1861] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:52.027921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c2b57-e9f0-401f-a9d3-abb9d7a18b5d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.042609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c2b57-e9f0-401f-a9d3-abb9d7a18b5d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.048122 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c572a-46ff-4805-872c-adf6819fbde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.048856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c572a-46ff-4805-872c-adf6819fbde1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.049294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c572a-46ff-4805-872c-adf6819fbde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.049573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c572a-46ff-4805-872c-adf6819fbde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.050648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c572a-46ff-4805-872c-adf6819fbde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.053458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c572a-46ff-4805-872c-adf6819fbde1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.065447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 928/1862] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:52.067560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e5276-5b68-4b9e-a8cc-7ac77fc0a246 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.071370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 935/1863] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:52.072521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c572a-46ff-4805-872c-adf6819fbde1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.082194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e5276-5b68-4b9e-a8cc-7ac77fc0a246 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.083956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c572a-46ff-4805-872c-adf6819fbde1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.087854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5213b2-d772-4b66-9ef8-f29bb1af2098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.088604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5213b2-d772-4b66-9ef8-f29bb1af2098 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.088957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5213b2-d772-4b66-9ef8-f29bb1af2098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.089463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5213b2-d772-4b66-9ef8-f29bb1af2098 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.090285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5213b2-d772-4b66-9ef8-f29bb1af2098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.091430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a960f2-7e40-4d78-8e3f-36acc89f7fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.092321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a960f2-7e40-4d78-8e3f-36acc89f7fa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.092600 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5213b2-d772-4b66-9ef8-f29bb1af2098 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.092740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a960f2-7e40-4d78-8e3f-36acc89f7fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.092962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a960f2-7e40-4d78-8e3f-36acc89f7fa1 admin admin] PATH_INFO: `/v3/projects/8121ad932a184f699a69c4a0b5ba74e6/users/0c0ba63865994106b3e23f8135388ead/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.114617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a960f2-7e40-4d78-8e3f-36acc89f7fa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8121ad932a184f699a69c4a0b5ba74e6, user_id=0c0ba63865994106b3e23f8135388ead, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.118561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a960f2-7e40-4d78-8e3f-36acc89f7fa1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.121767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:52.136814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 929/1864] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:52 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:52.137975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5213b2-d772-4b66-9ef8-f29bb1af2098 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.143633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:52.151628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 936/1865] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:52 2022] PUT /identity/v3/projects/8121ad932a184f699a69c4a0b5ba74e6/users/0c0ba63865994106b3e23f8135388ead/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:52.162507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba80096-3e1a-4d95-969c-bc2e9f08aac4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.162666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba80096-3e1a-4d95-969c-bc2e9f08aac4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.163404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba80096-3e1a-4d95-969c-bc2e9f08aac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.202955 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8ba80096-3e1a-4d95-969c-bc2e9f08aac4 None None] MFA Rules not processed for user `0c0ba63865994106b3e23f8135388ead`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:52.204323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8ba80096-3e1a-4d95-969c-bc2e9f08aac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:52.273052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5213b2-d772-4b66-9ef8-f29bb1af2098 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.281613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 937/1866] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:52.287024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba80096-3e1a-4d95-969c-bc2e9f08aac4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.361778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f49926-9b37-4a73-932a-4cc84756cbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.362471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f49926-9b37-4a73-932a-4cc84756cbcb admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.362754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f49926-9b37-4a73-932a-4cc84756cbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.363164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f49926-9b37-4a73-932a-4cc84756cbcb admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.364252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f49926-9b37-4a73-932a-4cc84756cbcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.368043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f49926-9b37-4a73-932a-4cc84756cbcb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.374331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba80096-3e1a-4d95-969c-bc2e9f08aac4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.381911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f4c4b-24b7-421e-9013-f0842d09dea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.383279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f4c4b-24b7-421e-9013-f0842d09dea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.383738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f4c4b-24b7-421e-9013-f0842d09dea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.384389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f4c4b-24b7-421e-9013-f0842d09dea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.384772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f4c4b-24b7-421e-9013-f0842d09dea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.387006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f4c4b-24b7-421e-9013-f0842d09dea5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.402835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 930/1867] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:52 2022] POST /identity/v3/users => generated 296 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:52.407417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f49926-9b37-4a73-932a-4cc84756cbcb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.424353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 938/1868] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:52 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:52.427914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f4c4b-24b7-421e-9013-f0842d09dea5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.434222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f49926-9b37-4a73-932a-4cc84756cbcb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.502892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f4c4b-24b7-421e-9013-f0842d09dea5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.506770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a79bc-f1e7-4acd-8e67-7da8ec07f197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.507490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a79bc-f1e7-4acd-8e67-7da8ec07f197 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.516304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a79bc-f1e7-4acd-8e67-7da8ec07f197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.516458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67542402-a530-4590-a097-742e8436eae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.516585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a79bc-f1e7-4acd-8e67-7da8ec07f197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.516692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67542402-a530-4590-a097-742e8436eae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.516692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67542402-a530-4590-a097-742e8436eae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.516850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a79bc-f1e7-4acd-8e67-7da8ec07f197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.516934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67542402-a530-4590-a097-742e8436eae7 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.516934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67542402-a530-4590-a097-742e8436eae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.517094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a79bc-f1e7-4acd-8e67-7da8ec07f197 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.517177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67542402-a530-4590-a097-742e8436eae7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.522369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 931/1869] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:52.525550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a79bc-f1e7-4acd-8e67-7da8ec07f197 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.530717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:52.537624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a79bc-f1e7-4acd-8e67-7da8ec07f197 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.538935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 939/1870] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:52 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:52.543045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67542402-a530-4590-a097-742e8436eae7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.546339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635c37b-5e38-4507-adfc-07b04374e7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.547843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635c37b-5e38-4507-adfc-07b04374e7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.548202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635c37b-5e38-4507-adfc-07b04374e7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.548487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2635c37b-5e38-4507-adfc-07b04374e7e4 admin admin] PATH_INFO: `/v3/projects/b32c15b56c8b4f10bee920d1ea932fa5/users/3188916bf1404f2a9bac0164bab42dd9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.554390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67542402-a530-4590-a097-742e8436eae7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.559747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67845570-0103-486e-97e0-9efb2d17f872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.560503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67845570-0103-486e-97e0-9efb2d17f872 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.560784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67845570-0103-486e-97e0-9efb2d17f872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.561180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67845570-0103-486e-97e0-9efb2d17f872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.562067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67845570-0103-486e-97e0-9efb2d17f872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.563743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67845570-0103-486e-97e0-9efb2d17f872 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.567202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635c37b-5e38-4507-adfc-07b04374e7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32c15b56c8b4f10bee920d1ea932fa5, user_id=3188916bf1404f2a9bac0164bab42dd9, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.570642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2635c37b-5e38-4507-adfc-07b04374e7e4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.576088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 940/1871] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:34:52.580348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67845570-0103-486e-97e0-9efb2d17f872 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.587010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 932/1872] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:52 2022] PUT /identity/v3/projects/b32c15b56c8b4f10bee920d1ea932fa5/users/3188916bf1404f2a9bac0164bab42dd9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:52.590569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67845570-0103-486e-97e0-9efb2d17f872 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.590757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635c37b-5e38-4507-adfc-07b04374e7e4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.603246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:52.651782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86123fa-4823-4775-a75d-181136e4acd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.652699 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86123fa-4823-4775-a75d-181136e4acd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.652979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86123fa-4823-4775-a75d-181136e4acd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.653256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86123fa-4823-4775-a75d-181136e4acd1 admin admin] PATH_INFO: `/v3/projects/18f340ef2c3945d1910cebdff34ad625/users/d98da21f0f7f42559eaa2dd12689d863/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.659813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2635c37b-5e38-4507-adfc-07b04374e7e4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.665629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa984e2-2c25-461c-8f3d-56bec9269ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.666299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa984e2-2c25-461c-8f3d-56bec9269ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.666584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa984e2-2c25-461c-8f3d-56bec9269ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.666885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa984e2-2c25-461c-8f3d-56bec9269ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.667623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa984e2-2c25-461c-8f3d-56bec9269ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.668971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86123fa-4823-4775-a75d-181136e4acd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f340ef2c3945d1910cebdff34ad625, user_id=d98da21f0f7f42559eaa2dd12689d863, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.669231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa984e2-2c25-461c-8f3d-56bec9269ab9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.671988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86123fa-4823-4775-a75d-181136e4acd1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.681137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 933/1873] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:34:52.685494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa984e2-2c25-461c-8f3d-56bec9269ab9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.687018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 941/1874] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:52 2022] PUT /identity/v3/projects/18f340ef2c3945d1910cebdff34ad625/users/d98da21f0f7f42559eaa2dd12689d863/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:52.690703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86123fa-4823-4775-a75d-181136e4acd1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.695775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa984e2-2c25-461c-8f3d-56bec9269ab9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.755794 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e15a1c-7bd0-4d54-878a-b49f74e368bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.756740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e15a1c-7bd0-4d54-878a-b49f74e368bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.757058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e15a1c-7bd0-4d54-878a-b49f74e368bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.757376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e15a1c-7bd0-4d54-878a-b49f74e368bb admin admin] PATH_INFO: `/v3/projects/b32c15b56c8b4f10bee920d1ea932fa5/users/3188916bf1404f2a9bac0164bab42dd9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.759668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e15a1c-7bd0-4d54-878a-b49f74e368bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32c15b56c8b4f10bee920d1ea932fa5, user_id=3188916bf1404f2a9bac0164bab42dd9, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.761476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86123fa-4823-4775-a75d-181136e4acd1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.762473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e15a1c-7bd0-4d54-878a-b49f74e368bb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.769245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a7668-9923-4c20-be02-0cdd15995cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.770135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a7668-9923-4c20-be02-0cdd15995cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.770484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a7668-9923-4c20-be02-0cdd15995cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.771749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a7668-9923-4c20-be02-0cdd15995cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.772846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a7668-9923-4c20-be02-0cdd15995cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.774396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a7668-9923-4c20-be02-0cdd15995cb2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.783863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 934/1875] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:52 2022] PUT /identity/v3/projects/b32c15b56c8b4f10bee920d1ea932fa5/users/3188916bf1404f2a9bac0164bab42dd9/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:52.785700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e3f5-4cce-46ff-8ce6-955524436596 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.786090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e3f5-4cce-46ff-8ce6-955524436596 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.786465 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e3f5-4cce-46ff-8ce6-955524436596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.786774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 942/1876] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:34:52.791004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a7668-9923-4c20-be02-0cdd15995cb2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.803004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a7668-9923-4c20-be02-0cdd15995cb2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:52.820275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6077e3f5-4cce-46ff-8ce6-955524436596 None None] MFA Rules not processed for user `3188916bf1404f2a9bac0164bab42dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:52.821510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6077e3f5-4cce-46ff-8ce6-955524436596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:52.873367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcbb3c-1f8d-4b41-ae49-e2be17ee3762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:52.874308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcbb3c-1f8d-4b41-ae49-e2be17ee3762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.874590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcbb3c-1f8d-4b41-ae49-e2be17ee3762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.874873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcbb3c-1f8d-4b41-ae49-e2be17ee3762 admin admin] PATH_INFO: `/v3/projects/18f340ef2c3945d1910cebdff34ad625/users/d98da21f0f7f42559eaa2dd12689d863/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.879784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcbb3c-1f8d-4b41-ae49-e2be17ee3762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f340ef2c3945d1910cebdff34ad625, user_id=d98da21f0f7f42559eaa2dd12689d863, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:52.882593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcbb3c-1f8d-4b41-ae49-e2be17ee3762 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:52.891387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:52.897706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 943/1877] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:52 2022] PUT /identity/v3/projects/18f340ef2c3945d1910cebdff34ad625/users/d98da21f0f7f42559eaa2dd12689d863/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:52.900246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 935/1878] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:52.914016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e6933-12e8-409b-88e4-8ab577d38e8d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:52.914582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e6933-12e8-409b-88e4-8ab577d38e8d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:52.914721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077e3f5-4cce-46ff-8ce6-955524436596 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:52.915156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e6933-12e8-409b-88e4-8ab577d38e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:52.956992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e64e6933-12e8-409b-88e4-8ab577d38e8d None None] MFA Rules not processed for user `d98da21f0f7f42559eaa2dd12689d863`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:52.958577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e64e6933-12e8-409b-88e4-8ab577d38e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:52.976706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:53.010529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077e3f5-4cce-46ff-8ce6-955524436596 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:53.078844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 944/1879] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:53.087892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb0a27-899d-4760-a7c6-a40a3ced570b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:53.089494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb0a27-899d-4760-a7c6-a40a3ced570b service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:53.089630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e6933-12e8-409b-88e4-8ab577d38e8d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:53.089726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb0a27-899d-4760-a7c6-a40a3ced570b service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:53.089726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb0a27-899d-4760-a7c6-a40a3ced570b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:53.091322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e64e6933-12e8-409b-88e4-8ab577d38e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:53.104588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e6933-12e8-409b-88e4-8ab577d38e8d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:53.108690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c3fdc-9656-485f-b9ab-dcb0be9e5a13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:53.109500 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c3fdc-9656-485f-b9ab-dcb0be9e5a13 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:53.109946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c3fdc-9656-485f-b9ab-dcb0be9e5a13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:53.110291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c3fdc-9656-485f-b9ab-dcb0be9e5a13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:53.129792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c3fdc-9656-485f-b9ab-dcb0be9e5a13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:53.138044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c3fdc-9656-485f-b9ab-dcb0be9e5a13 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:53.154294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 945/1880] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:53.183239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb0a27-899d-4760-a7c6-a40a3ced570b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:53.184953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb0a27-899d-4760-a7c6-a40a3ced570b service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:53.224256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 936/1881] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:53.290503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c3fdc-9656-485f-b9ab-dcb0be9e5a13 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:53.303424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c3fdc-9656-485f-b9ab-dcb0be9e5a13 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:53.312484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aebf22-bfe0-45f5-b04c-4c7f656b014d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:53.313615 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aebf22-bfe0-45f5-b04c-4c7f656b014d service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:53.314065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aebf22-bfe0-45f5-b04c-4c7f656b014d service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:53.314493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aebf22-bfe0-45f5-b04c-4c7f656b014d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:53.400536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aebf22-bfe0-45f5-b04c-4c7f656b014d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:53.401913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aebf22-bfe0-45f5-b04c-4c7f656b014d service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:53.415438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 946/1882] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:53.799006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb0a27-899d-4760-a7c6-a40a3ced570b service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:53.812908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb0a27-899d-4760-a7c6-a40a3ced570b service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:53.823125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c7208-e890-4726-8249-f066b59b7b16 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd98da21f0f7f42559eaa2dd12689d863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f340ef2c3945d1910cebdff34ad625', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:53.824417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c7208-e890-4726-8249-f066b59b7b16 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:53.824926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c7208-e890-4726-8249-f066b59b7b16 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:53.825355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c7208-e890-4726-8249-f066b59b7b16 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:53.826220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c7208-e890-4726-8249-f066b59b7b16 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:53.827362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c7208-e890-4726-8249-f066b59b7b16 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:53.829120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c7208-e890-4726-8249-f066b59b7b16 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:53.869176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 937/1883] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:53.878889 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aebf22-bfe0-45f5-b04c-4c7f656b014d service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:53.893468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aebf22-bfe0-45f5-b04c-4c7f656b014d service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:53.903210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717a08b-5786-4077-847b-f5183ddcabc2 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd98da21f0f7f42559eaa2dd12689d863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f340ef2c3945d1910cebdff34ad625', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:53.903983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717a08b-5786-4077-847b-f5183ddcabc2 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:53.904376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717a08b-5786-4077-847b-f5183ddcabc2 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:53.904722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717a08b-5786-4077-847b-f5183ddcabc2 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:53.905612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717a08b-5786-4077-847b-f5183ddcabc2 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:53.915347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717a08b-5786-4077-847b-f5183ddcabc2 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:53.973895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 947/1884] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:53 2022] POST /identity/v3/projects => generated 355 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:53.978960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c7208-e890-4726-8249-f066b59b7b16 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:54.000547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c7208-e890-4726-8249-f066b59b7b16 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:54.094072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd98da21f0f7f42559eaa2dd12689d863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f340ef2c3945d1910cebdff34ad625', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:54.094940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:54.095393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:54.095679 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:54.096455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:54.096711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:54.098453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:54.117761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 938/1885] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:54.169051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717a08b-5786-4077-847b-f5183ddcabc2 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:54.180557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717a08b-5786-4077-847b-f5183ddcabc2 tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:54.185250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd98da21f0f7f42559eaa2dd12689d863', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f340ef2c3945d1910cebdff34ad625', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:54.186122 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:54.186413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:54.186691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] PATH_INFO: `/v3/projects/8cadf69813dd4d3489d1396c60c8561b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:54.187950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8cadf69813dd4d3489d1396c60c8561b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:54.189880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:54.241739 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:54.246075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cadf69813dd4d3489d1396c60c8561b', 'request_id': 'req-47ac4df5-767a-4ce4-913a-3eb6410c81ab'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.252733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cadf69813dd4d3489d1396c60c8561b', 'request_id': 'req-47ac4df5-767a-4ce4-913a-3eb6410c81ab'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.259387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cadf69813dd4d3489d1396c60c8561b', 'request_id': 'req-47ac4df5-767a-4ce4-913a-3eb6410c81ab'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.274035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] The token cache is being invalidate because project 8cadf69813dd4d3489d1396c60c8561b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:54.274352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.275063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47ac4df5-767a-4ce4-913a-3eb6410c81ab tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.276833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 948/1886] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:54 2022] DELETE /identity/v3/projects/8cadf69813dd4d3489d1396c60c8561b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:54.484058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:54.485850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:54.500119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:54.555227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10464d4-e7b4-44e6-96bf-694a1bd2ee8d tempest-VolumeQuotaClassesTest-1077672017 tempest-VolumeQuotaClassesTest-1077672017-project-admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:54.615618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:54.616532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:54.616814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:54.617091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] PATH_INFO: `/v3/users/0c0ba63865994106b3e23f8135388ead` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:54.618350 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0ba63865994106b3e23f8135388ead)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:54.620323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:54.674581 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0ba63865994106b3e23f8135388ead'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.675149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0ba63865994106b3e23f8135388ead'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.684629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0ba63865994106b3e23f8135388ead'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.685238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0ba63865994106b3e23f8135388ead'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.699538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0ba63865994106b3e23f8135388ead'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:54.715155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 939/1887] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:54 2022] DELETE /identity/v3/users/0c0ba63865994106b3e23f8135388ead => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:54.719582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5b13e-89a3-4cf8-81bc-28c11076fafe None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:54.719870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5b13e-89a3-4cf8-81bc-28c11076fafe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:54.720148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5b13e-89a3-4cf8-81bc-28c11076fafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:54.738352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4ee5b13e-89a3-4cf8-81bc-28c11076fafe None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:54.739628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4ee5b13e-89a3-4cf8-81bc-28c11076fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:54.806106 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 949/1888] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:54.815728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:54.817396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:54.863340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:54.884965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8cb2d7-9bda-40ae-b03f-2af2e6869b20 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:54.890124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ce0b2-19b2-4d57-aab2-123cea81c135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:54.890715 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ce0b2-19b2-4d57-aab2-123cea81c135 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:54.891006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ce0b2-19b2-4d57-aab2-123cea81c135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:54.891286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ce0b2-19b2-4d57-aab2-123cea81c135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:54.902728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ce0b2-19b2-4d57-aab2-123cea81c135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:54.904114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ce0b2-19b2-4d57-aab2-123cea81c135 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:54.916532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 940/1889] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:55.224681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5b13e-89a3-4cf8-81bc-28c11076fafe None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:55.226281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4ee5b13e-89a3-4cf8-81bc-28c11076fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:55.240424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5b13e-89a3-4cf8-81bc-28c11076fafe None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:55.245614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:55.246557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:55.246850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:55.247135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] PATH_INFO: `/v3/projects/8121ad932a184f699a69c4a0b5ba74e6` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:55.248375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8121ad932a184f699a69c4a0b5ba74e6)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:55.250303 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:55.299345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8121ad932a184f699a69c4a0b5ba74e6', 'request_id': 'req-5135eea0-0c00-43d9-bb97-795c38148c12'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.306840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8121ad932a184f699a69c4a0b5ba74e6', 'request_id': 'req-5135eea0-0c00-43d9-bb97-795c38148c12'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.313432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8121ad932a184f699a69c4a0b5ba74e6', 'request_id': 'req-5135eea0-0c00-43d9-bb97-795c38148c12'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.329133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] The token cache is being invalidate because project 8121ad932a184f699a69c4a0b5ba74e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:55.329418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.330138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.331736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 950/1890] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:55 2022] DELETE /identity/v3/projects/8121ad932a184f699a69c4a0b5ba74e6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:55.335777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ce0b2-19b2-4d57-aab2-123cea81c135 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:55.345446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ce0b2-19b2-4d57-aab2-123cea81c135 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:55.405883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:55.406787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:55.407067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:55.407371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] PATH_INFO: `/v3/users/d98da21f0f7f42559eaa2dd12689d863` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:55.408730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98da21f0f7f42559eaa2dd12689d863)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:55.410698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:55.464258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98da21f0f7f42559eaa2dd12689d863'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.464832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98da21f0f7f42559eaa2dd12689d863'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.473837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98da21f0f7f42559eaa2dd12689d863'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.474542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98da21f0f7f42559eaa2dd12689d863'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.487956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c960dcde-7341-43b6-9314-dc9113641e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98da21f0f7f42559eaa2dd12689d863'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.500560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 941/1891] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:55 2022] DELETE /identity/v3/users/d98da21f0f7f42559eaa2dd12689d863 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:55.774572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:55.776031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:55.789156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5135eea0-0c00-43d9-bb97-795c38148c12 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:55.839884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:55.848405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:55.849361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:55.849860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:55.850513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] PATH_INFO: `/v3/projects/18f340ef2c3945d1910cebdff34ad625` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:55.851837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f340ef2c3945d1910cebdff34ad625)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:55.854292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:55.900206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f340ef2c3945d1910cebdff34ad625', 'request_id': 'req-cdfe62c0-158d-44bc-9c80-838529158be1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.905739 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f340ef2c3945d1910cebdff34ad625', 'request_id': 'req-cdfe62c0-158d-44bc-9c80-838529158be1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.912151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f340ef2c3945d1910cebdff34ad625', 'request_id': 'req-cdfe62c0-158d-44bc-9c80-838529158be1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.927730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] The token cache is being invalidate because project 18f340ef2c3945d1910cebdff34ad625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:55.928019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.928798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cdfe62c0-158d-44bc-9c80-838529158be1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:55.930532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 951/1892] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:55 2022] DELETE /identity/v3/projects/18f340ef2c3945d1910cebdff34ad625 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:55.936479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0a790-35c2-4417-9e42-6dfdf093bc30 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:55.939733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0a790-35c2-4417-9e42-6dfdf093bc30 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:55.940205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0a790-35c2-4417-9e42-6dfdf093bc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:55.978687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-21c0a790-35c2-4417-9e42-6dfdf093bc30 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:55.979960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-21c0a790-35c2-4417-9e42-6dfdf093bc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:56.053392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 942/1893] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:56.061338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cf1cc-8a04-45f0-af3b-b2118ce242ab None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.061338 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cf1cc-8a04-45f0-af3b-b2118ce242ab None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.061602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cf1cc-8a04-45f0-af3b-b2118ce242ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.081859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-df9cf1cc-8a04-45f0-af3b-b2118ce242ab None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:56.083058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-df9cf1cc-8a04-45f0-af3b-b2118ce242ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:56.091964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 952/1894] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:56.096164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0a790-35c2-4417-9e42-6dfdf093bc30 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.102452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:56.108333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0a790-35c2-4417-9e42-6dfdf093bc30 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.113613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79aa054-8bb8-4157-b3e1-19f364fc9ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.114448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79aa054-8bb8-4157-b3e1-19f364fc9ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.115340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79aa054-8bb8-4157-b3e1-19f364fc9ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.115340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79aa054-8bb8-4157-b3e1-19f364fc9ba9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.116267 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79aa054-8bb8-4157-b3e1-19f364fc9ba9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:34:56.116916 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79aa054-8bb8-4157-b3e1-19f364fc9ba9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.118962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79aa054-8bb8-4157-b3e1-19f364fc9ba9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.135582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 943/1895] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:34:56.137237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cf1cc-8a04-45f0-af3b-b2118ce242ab None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.146541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cf1cc-8a04-45f0-af3b-b2118ce242ab None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.151164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f07b6-88d7-4172-8fc1-72bff6e6df7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.151851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f07b6-88d7-4172-8fc1-72bff6e6df7c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.152143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f07b6-88d7-4172-8fc1-72bff6e6df7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.152403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f07b6-88d7-4172-8fc1-72bff6e6df7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.153096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f07b6-88d7-4172-8fc1-72bff6e6df7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.155036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f07b6-88d7-4172-8fc1-72bff6e6df7c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.192971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 953/1896] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:56 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:56.194026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79aa054-8bb8-4157-b3e1-19f364fc9ba9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.205558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79aa054-8bb8-4157-b3e1-19f364fc9ba9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.297577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff18b64-f641-4509-bc8d-c529d711f39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.298287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff18b64-f641-4509-bc8d-c529d711f39f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.298573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff18b64-f641-4509-bc8d-c529d711f39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.298853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff18b64-f641-4509-bc8d-c529d711f39f admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.299669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff18b64-f641-4509-bc8d-c529d711f39f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.301714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff18b64-f641-4509-bc8d-c529d711f39f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.326654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 944/1897] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:56 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:56.328025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f07b6-88d7-4172-8fc1-72bff6e6df7c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.338687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f07b6-88d7-4172-8fc1-72bff6e6df7c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.343477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9fbeb-91e5-4d75-9742-cf57e0955ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.344216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9fbeb-91e5-4d75-9742-cf57e0955ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.344486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9fbeb-91e5-4d75-9742-cf57e0955ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.344880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9fbeb-91e5-4d75-9742-cf57e0955ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.345604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9fbeb-91e5-4d75-9742-cf57e0955ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.347245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9fbeb-91e5-4d75-9742-cf57e0955ca1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.358348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 954/1898] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:34:56.362622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff18b64-f641-4509-bc8d-c529d711f39f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.374512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff18b64-f641-4509-bc8d-c529d711f39f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.380479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7a4a8-2eb9-4529-a3c7-3210e5943b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.381176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7a4a8-2eb9-4529-a3c7-3210e5943b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.381455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7a4a8-2eb9-4529-a3c7-3210e5943b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.381793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7a4a8-2eb9-4529-a3c7-3210e5943b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.382505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7a4a8-2eb9-4529-a3c7-3210e5943b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.384228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7a4a8-2eb9-4529-a3c7-3210e5943b02 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.394658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 945/1899] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:56.399138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9fbeb-91e5-4d75-9742-cf57e0955ca1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.409763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9fbeb-91e5-4d75-9742-cf57e0955ca1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.415250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c00cc-8766-4160-8042-d566328b638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.416396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c00cc-8766-4160-8042-d566328b638f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.416676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c00cc-8766-4160-8042-d566328b638f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.416951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c00cc-8766-4160-8042-d566328b638f admin admin] PATH_INFO: `/v3/projects/35dbd2c8211e4ccf95607e3f6a329294/users/aef8f0a7bacc4f6b9bb6ccf8e4d86ecd/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.433752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c00cc-8766-4160-8042-d566328b638f admin admin] RBAC: Authorizing `identity:create_grant(project_id=35dbd2c8211e4ccf95607e3f6a329294, user_id=aef8f0a7bacc4f6b9bb6ccf8e4d86ecd, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.436473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c00cc-8766-4160-8042-d566328b638f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.451369 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 955/1900] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:56 2022] PUT /identity/v3/projects/35dbd2c8211e4ccf95607e3f6a329294/users/aef8f0a7bacc4f6b9bb6ccf8e4d86ecd/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:56.464201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3f2f7-0293-4480-ac16-94fbdca65700 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.464505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3f2f7-0293-4480-ac16-94fbdca65700 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.464807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3f2f7-0293-4480-ac16-94fbdca65700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.493637 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e4c3f2f7-0293-4480-ac16-94fbdca65700 None None] MFA Rules not processed for user `aef8f0a7bacc4f6b9bb6ccf8e4d86ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:56.494847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e4c3f2f7-0293-4480-ac16-94fbdca65700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:56.573820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 946/1901] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:34:56.578547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c00cc-8766-4160-8042-d566328b638f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.659380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c00cc-8766-4160-8042-d566328b638f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.665849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f8830-7bd0-4af5-972e-13fe32222319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.666528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f8830-7bd0-4af5-972e-13fe32222319 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.666809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f8830-7bd0-4af5-972e-13fe32222319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.667084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f8830-7bd0-4af5-972e-13fe32222319 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.667812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f8830-7bd0-4af5-972e-13fe32222319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.669741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f8830-7bd0-4af5-972e-13fe32222319 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.704650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 956/1902] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:56 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:56.709587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3f2f7-0293-4480-ac16-94fbdca65700 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.720354 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3f2f7-0293-4480-ac16-94fbdca65700 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.782099 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:56.790551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f976ec-d7d6-4b5f-b54c-7bea608c9c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.791314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f976ec-d7d6-4b5f-b54c-7bea608c9c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.791611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f976ec-d7d6-4b5f-b54c-7bea608c9c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.791899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f976ec-d7d6-4b5f-b54c-7bea608c9c9e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.792622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f976ec-d7d6-4b5f-b54c-7bea608c9c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.794941 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f976ec-d7d6-4b5f-b54c-7bea608c9c9e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.820039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 947/1903] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:56 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:56.825332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f8830-7bd0-4af5-972e-13fe32222319 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.837284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f8830-7bd0-4af5-972e-13fe32222319 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.843332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aafc3-71f5-49ab-b83b-1fd55de34761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.844035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aafc3-71f5-49ab-b83b-1fd55de34761 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.844312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aafc3-71f5-49ab-b83b-1fd55de34761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.844671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619aafc3-71f5-49ab-b83b-1fd55de34761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.845397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aafc3-71f5-49ab-b83b-1fd55de34761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.847004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619aafc3-71f5-49ab-b83b-1fd55de34761 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.848510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:56.861412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 957/1904] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:56.865776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f976ec-d7d6-4b5f-b54c-7bea608c9c9e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.882532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f976ec-d7d6-4b5f-b54c-7bea608c9c9e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.888513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33449112-fae2-4bf6-befa-38ef152e19f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.889178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33449112-fae2-4bf6-befa-38ef152e19f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.889456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33449112-fae2-4bf6-befa-38ef152e19f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.889726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33449112-fae2-4bf6-befa-38ef152e19f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.890440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33449112-fae2-4bf6-befa-38ef152e19f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.892092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33449112-fae2-4bf6-befa-38ef152e19f8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.903184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 948/1905] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:56.908127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aafc3-71f5-49ab-b83b-1fd55de34761 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:56.922805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619aafc3-71f5-49ab-b83b-1fd55de34761 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:56.932172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055d33e-1d4e-4991-a035-2fdf9cd8868b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:56.933585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055d33e-1d4e-4991-a035-2fdf9cd8868b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.934050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055d33e-1d4e-4991-a035-2fdf9cd8868b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.934498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055d33e-1d4e-4991-a035-2fdf9cd8868b admin admin] PATH_INFO: `/v3/projects/5b36792942584f5289876f74e9619452/users/b598bb94abd24e29b2e56e897505e7ec/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:56.959147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055d33e-1d4e-4991-a035-2fdf9cd8868b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b36792942584f5289876f74e9619452, user_id=b598bb94abd24e29b2e56e897505e7ec, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:56.962369 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055d33e-1d4e-4991-a035-2fdf9cd8868b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:56.980738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 958/1906] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:56 2022] PUT /identity/v3/projects/5b36792942584f5289876f74e9619452/users/b598bb94abd24e29b2e56e897505e7ec/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:56.996708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ac51c-5fac-4f84-b474-a99dc008330e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:56.997483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ac51c-5fac-4f84-b474-a99dc008330e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:56.997483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ac51c-5fac-4f84-b474-a99dc008330e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:57.028692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3b9ac51c-5fac-4f84-b474-a99dc008330e None None] MFA Rules not processed for user `b598bb94abd24e29b2e56e897505e7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:57.030085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3b9ac51c-5fac-4f84-b474-a99dc008330e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:57.117081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 949/1907] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:57.122421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055d33e-1d4e-4991-a035-2fdf9cd8868b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:57.197437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055d33e-1d4e-4991-a035-2fdf9cd8868b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:57.205828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b720f-c98e-419c-8ae0-10c7e202b60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:57.207306 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b720f-c98e-419c-8ae0-10c7e202b60d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:57.207872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b720f-c98e-419c-8ae0-10c7e202b60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:57.208403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b720f-c98e-419c-8ae0-10c7e202b60d admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:57.209663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b720f-c98e-419c-8ae0-10c7e202b60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:57.211949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b720f-c98e-419c-8ae0-10c7e202b60d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:57.254755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 959/1908] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:34:57 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:34:57.259089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ac51c-5fac-4f84-b474-a99dc008330e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:57.271825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ac51c-5fac-4f84-b474-a99dc008330e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:57.335624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57785fd-51a9-439f-9526-b36f56cdc325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:57.336511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57785fd-51a9-439f-9526-b36f56cdc325 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:57.336806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57785fd-51a9-439f-9526-b36f56cdc325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:57.337079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57785fd-51a9-439f-9526-b36f56cdc325 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:57.337886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57785fd-51a9-439f-9526-b36f56cdc325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:57.340340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57785fd-51a9-439f-9526-b36f56cdc325 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:57.363231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 950/1909] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:34:57 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:34:57.367095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b720f-c98e-419c-8ae0-10c7e202b60d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:57.379553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b720f-c98e-419c-8ae0-10c7e202b60d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:57.385123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0944-3ab6-4586-923e-99d5f98a811c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:57.385883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0944-3ab6-4586-923e-99d5f98a811c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:57.386161 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0944-3ab6-4586-923e-99d5f98a811c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:57.386435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c0944-3ab6-4586-923e-99d5f98a811c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:57.387157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c0944-3ab6-4586-923e-99d5f98a811c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:57.388848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c0944-3ab6-4586-923e-99d5f98a811c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:57.400654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 960/1910] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:57.404780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57785fd-51a9-439f-9526-b36f56cdc325 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:57.415200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57785fd-51a9-439f-9526-b36f56cdc325 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:57.425802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73664042-f30b-4401-90df-b1121ad642df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:57.426878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73664042-f30b-4401-90df-b1121ad642df admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:57.427260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73664042-f30b-4401-90df-b1121ad642df admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:57.427729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73664042-f30b-4401-90df-b1121ad642df admin admin] PATH_INFO: `/v3/projects/709ebb39f1084a27b66caa7ec1546d8c/users/1513689f654e4cba8316819091af23ca/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:57.458384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73664042-f30b-4401-90df-b1121ad642df admin admin] RBAC: Authorizing `identity:create_grant(project_id=709ebb39f1084a27b66caa7ec1546d8c, user_id=1513689f654e4cba8316819091af23ca, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:57.461442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73664042-f30b-4401-90df-b1121ad642df admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:57.479332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 951/1911] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:57 2022] PUT /identity/v3/projects/709ebb39f1084a27b66caa7ec1546d8c/users/1513689f654e4cba8316819091af23ca/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:34:57.483398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0944-3ab6-4586-923e-99d5f98a811c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:57.494028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c0944-3ab6-4586-923e-99d5f98a811c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:57.565703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c244210-55c6-48f2-a099-c4efdd73c6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:57.566457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c244210-55c6-48f2-a099-c4efdd73c6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:57.566803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c244210-55c6-48f2-a099-c4efdd73c6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:57.567142 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c244210-55c6-48f2-a099-c4efdd73c6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:57.568024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c244210-55c6-48f2-a099-c4efdd73c6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:57.569888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c244210-55c6-48f2-a099-c4efdd73c6d5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:57.584452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 961/1912] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:34:57.585966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73664042-f30b-4401-90df-b1121ad642df admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:57.596503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73664042-f30b-4401-90df-b1121ad642df admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:57.601656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a537d-5e72-4067-93bc-f5359cca8220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:57.602621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a537d-5e72-4067-93bc-f5359cca8220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:57.602921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a537d-5e72-4067-93bc-f5359cca8220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:57.603176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a537d-5e72-4067-93bc-f5359cca8220 admin admin] PATH_INFO: `/v3/projects/709ebb39f1084a27b66caa7ec1546d8c/users/1513689f654e4cba8316819091af23ca/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:57.605167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a537d-5e72-4067-93bc-f5359cca8220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=709ebb39f1084a27b66caa7ec1546d8c, user_id=1513689f654e4cba8316819091af23ca, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:57.607851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a537d-5e72-4067-93bc-f5359cca8220 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:57.618648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 952/1913] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:34:57 2022] PUT /identity/v3/projects/709ebb39f1084a27b66caa7ec1546d8c/users/1513689f654e4cba8316819091af23ca/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:57.628931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4894e3f-05ca-4de5-b7f9-9d865e89bd93 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:57.629220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4894e3f-05ca-4de5-b7f9-9d865e89bd93 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:57.629499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4894e3f-05ca-4de5-b7f9-9d865e89bd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:57.655112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b4894e3f-05ca-4de5-b7f9-9d865e89bd93 None None] MFA Rules not processed for user `1513689f654e4cba8316819091af23ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:57.656325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b4894e3f-05ca-4de5-b7f9-9d865e89bd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:57.687666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:57.731445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 962/1914] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:57.743219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a537d-5e72-4067-93bc-f5359cca8220 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:57.745204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2d4a537d-5e72-4067-93bc-f5359cca8220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:57.771972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:57.811923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a537d-5e72-4067-93bc-f5359cca8220 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:57.815976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f6794-ee7d-4773-93e6-24b0f57edbd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:57.816620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f6794-ee7d-4773-93e6-24b0f57edbd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:57.816910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f6794-ee7d-4773-93e6-24b0f57edbd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:57.817180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f6794-ee7d-4773-93e6-24b0f57edbd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:57.826848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f6794-ee7d-4773-93e6-24b0f57edbd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:57.828285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f6794-ee7d-4773-93e6-24b0f57edbd0 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:57.837915 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 953/1915] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:58.411286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4894e3f-05ca-4de5-b7f9-9d865e89bd93 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:58.413922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b4894e3f-05ca-4de5-b7f9-9d865e89bd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:58.427447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4894e3f-05ca-4de5-b7f9-9d865e89bd93 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:58.432152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576512f-5780-4549-b0db-84d151806a98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:58.432797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576512f-5780-4549-b0db-84d151806a98 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:58.433100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576512f-5780-4549-b0db-84d151806a98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:58.433375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576512f-5780-4549-b0db-84d151806a98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:58.500561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576512f-5780-4549-b0db-84d151806a98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:58.502325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576512f-5780-4549-b0db-84d151806a98 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:58.513143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 963/1916] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:59.132017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f6794-ee7d-4773-93e6-24b0f57edbd0 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:59.133548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-aa6f6794-ee7d-4773-93e6-24b0f57edbd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:59.200703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f6794-ee7d-4773-93e6-24b0f57edbd0 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:59.206231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:59.207117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:59.207444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:59.207719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] PATH_INFO: `/v3/users/fe3c90fd557340c498c575824fdc7cdd` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:59.208912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3c90fd557340c498c575824fdc7cdd)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:59.211128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:59.264096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3c90fd557340c498c575824fdc7cdd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.264998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3c90fd557340c498c575824fdc7cdd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.273345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3c90fd557340c498c575824fdc7cdd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.273987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3c90fd557340c498c575824fdc7cdd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.285645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3c90fd557340c498c575824fdc7cdd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.298091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 954/1917] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:34:59 2022] DELETE /identity/v3/users/fe3c90fd557340c498c575824fdc7cdd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:59.302592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933d9e7-fc32-4ad6-80e3-a78452fce803 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:59.302881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933d9e7-fc32-4ad6-80e3-a78452fce803 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:59.303157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933d9e7-fc32-4ad6-80e3-a78452fce803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:59.320132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d933d9e7-fc32-4ad6-80e3-a78452fce803 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:59.321327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d933d9e7-fc32-4ad6-80e3-a78452fce803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:59.382800 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 964/1918] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:34:59.393591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:59.395627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:59.444628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:59.464078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34961fa4-a82b-4f63-b277-40b617533a5b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:59.467754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec07a3-30cd-4dfb-b2f8-b283d78d08c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:59.468415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec07a3-30cd-4dfb-b2f8-b283d78d08c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:59.468698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec07a3-30cd-4dfb-b2f8-b283d78d08c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:59.468972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec07a3-30cd-4dfb-b2f8-b283d78d08c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:59.480800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec07a3-30cd-4dfb-b2f8-b283d78d08c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:59.482724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec07a3-30cd-4dfb-b2f8-b283d78d08c7 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:59.495887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 955/1919] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:34:59.794029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933d9e7-fc32-4ad6-80e3-a78452fce803 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:34:59.796686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d933d9e7-fc32-4ad6-80e3-a78452fce803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:34:59.810143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933d9e7-fc32-4ad6-80e3-a78452fce803 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:34:59.816640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:34:59.817617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:59.818129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:59.818617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] PATH_INFO: `/v3/projects/58e6bc94f4d44deeb1341c8a74e5c5f4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:59.820711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=58e6bc94f4d44deeb1341c8a74e5c5f4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:34:59.822855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:34:59.878812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58e6bc94f4d44deeb1341c8a74e5c5f4', 'request_id': 'req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.885186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58e6bc94f4d44deeb1341c8a74e5c5f4', 'request_id': 'req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.887502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:34:59.892444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58e6bc94f4d44deeb1341c8a74e5c5f4', 'request_id': 'req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.908362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] The token cache is being invalidate because project 58e6bc94f4d44deeb1341c8a74e5c5f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:34:59.908721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.909433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02962cf0-1892-4b92-8d6a-f67d5d1f8c5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:34:59.911050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 965/1920] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:34:59 2022] DELETE /identity/v3/projects/58e6bc94f4d44deeb1341c8a74e5c5f4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:34:59.916863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695965-b50c-4bd1-9415-8c49dbca5407 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:34:59.917163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695965-b50c-4bd1-9415-8c49dbca5407 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:34:59.917451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695965-b50c-4bd1-9415-8c49dbca5407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:34:59.938098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1b695965-b50c-4bd1-9415-8c49dbca5407 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:34:59.939368 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b695965-b50c-4bd1-9415-8c49dbca5407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:00.046210 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 956/1921] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:00.054341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635670c-bbb9-4d77-9ca8-8e5572c31ae9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:00.055064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635670c-bbb9-4d77-9ca8-8e5572c31ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:00.055779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635670c-bbb9-4d77-9ca8-8e5572c31ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:00.090174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4635670c-bbb9-4d77-9ca8-8e5572c31ae9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:00.091798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4635670c-bbb9-4d77-9ca8-8e5572c31ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:00.103269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 966/1922] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:00.122733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695965-b50c-4bd1-9415-8c49dbca5407 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:00.137991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695965-b50c-4bd1-9415-8c49dbca5407 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:00.143960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda8be4-1752-4446-8e95-f743b9a0ee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:00.144685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda8be4-1752-4446-8e95-f743b9a0ee3d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:00.145001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda8be4-1752-4446-8e95-f743b9a0ee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:00.145404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda8be4-1752-4446-8e95-f743b9a0ee3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:00.146119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda8be4-1752-4446-8e95-f743b9a0ee3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:00.146913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda8be4-1752-4446-8e95-f743b9a0ee3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:00.149017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda8be4-1752-4446-8e95-f743b9a0ee3d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:00.166300 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 957/1923] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:00.168080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635670c-bbb9-4d77-9ca8-8e5572c31ae9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:00.183090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635670c-bbb9-4d77-9ca8-8e5572c31ae9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:00.209620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f0bc0-5bbd-45be-8847-cc5a14ac0057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:00.213084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f0bc0-5bbd-45be-8847-cc5a14ac0057 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:00.214984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f0bc0-5bbd-45be-8847-cc5a14ac0057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:00.216206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f0bc0-5bbd-45be-8847-cc5a14ac0057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:00.218690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f0bc0-5bbd-45be-8847-cc5a14ac0057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:00.225450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f0bc0-5bbd-45be-8847-cc5a14ac0057 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:00.267323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 967/1924] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:00 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:00.271443 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda8be4-1752-4446-8e95-f743b9a0ee3d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:00.283043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda8be4-1752-4446-8e95-f743b9a0ee3d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:00.352582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89d1b2-d0bf-4e32-968e-ab736bc9a6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:00.353399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89d1b2-d0bf-4e32-968e-ab736bc9a6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:00.353748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89d1b2-d0bf-4e32-968e-ab736bc9a6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:00.354132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89d1b2-d0bf-4e32-968e-ab736bc9a6d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:00.355093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89d1b2-d0bf-4e32-968e-ab736bc9a6d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:00.357252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb89d1b2-d0bf-4e32-968e-ab736bc9a6d6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:00.386483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 958/1925] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:00 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:00.390590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f0bc0-5bbd-45be-8847-cc5a14ac0057 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:00.403055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f0bc0-5bbd-45be-8847-cc5a14ac0057 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:00.418631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb9061-c234-40ab-bde0-20276439bd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:00.419814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb9061-c234-40ab-bde0-20276439bd2f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:00.420602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb9061-c234-40ab-bde0-20276439bd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:00.421320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb9061-c234-40ab-bde0-20276439bd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:00.422345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb9061-c234-40ab-bde0-20276439bd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:00.424288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb9061-c234-40ab-bde0-20276439bd2f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:00.440593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 968/1926] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:35:00.444383 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89d1b2-d0bf-4e32-968e-ab736bc9a6d6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:00.461086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89d1b2-d0bf-4e32-968e-ab736bc9a6d6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:00.467103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72874d2e-9526-431e-a78f-d9b06a521037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:00.467837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72874d2e-9526-431e-a78f-d9b06a521037 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:00.468165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72874d2e-9526-431e-a78f-d9b06a521037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:00.468615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72874d2e-9526-431e-a78f-d9b06a521037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:00.470111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72874d2e-9526-431e-a78f-d9b06a521037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:00.471926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72874d2e-9526-431e-a78f-d9b06a521037 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:00.487038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 959/1927] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:00.491704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb9061-c234-40ab-bde0-20276439bd2f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:00.507921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb9061-c234-40ab-bde0-20276439bd2f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:00.515919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e7a5c-2b4c-4c4b-8ce7-c8de4201a3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:00.517249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e7a5c-2b4c-4c4b-8ce7-c8de4201a3db admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:00.517989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e7a5c-2b4c-4c4b-8ce7-c8de4201a3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:00.518450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e7a5c-2b4c-4c4b-8ce7-c8de4201a3db admin admin] PATH_INFO: `/v3/projects/c5ba744f426b45c1ac17bc635db87532/users/46fb9dcfee35481099068771381f5227/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:00.543478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e7a5c-2b4c-4c4b-8ce7-c8de4201a3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5ba744f426b45c1ac17bc635db87532, user_id=46fb9dcfee35481099068771381f5227, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:00.547830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e7a5c-2b4c-4c4b-8ce7-c8de4201a3db admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:00.566542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 969/1928] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:00 2022] PUT /identity/v3/projects/c5ba744f426b45c1ac17bc635db87532/users/46fb9dcfee35481099068771381f5227/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:00.576701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c978c7f-847b-43af-8741-4beb192b7339 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:00.577006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c978c7f-847b-43af-8741-4beb192b7339 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:00.577277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c978c7f-847b-43af-8741-4beb192b7339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:00.610496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3c978c7f-847b-43af-8741-4beb192b7339 None None] MFA Rules not processed for user `46fb9dcfee35481099068771381f5227`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:00.611613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3c978c7f-847b-43af-8741-4beb192b7339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:00.661275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:00.694709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 960/1929] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:00.708148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e7a5c-2b4c-4c4b-8ce7-c8de4201a3db admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:00.710988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3a1e7a5c-2b4c-4c4b-8ce7-c8de4201a3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:00.790789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e7a5c-2b4c-4c4b-8ce7-c8de4201a3db admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:00.796985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00808a93-188a-48e2-a0cf-2bf03c73ed0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:00.797784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00808a93-188a-48e2-a0cf-2bf03c73ed0c service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:00.798076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00808a93-188a-48e2-a0cf-2bf03c73ed0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:00.798350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00808a93-188a-48e2-a0cf-2bf03c73ed0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:00.810938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00808a93-188a-48e2-a0cf-2bf03c73ed0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:00.812658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00808a93-188a-48e2-a0cf-2bf03c73ed0c service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:00.825867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 970/1930] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:35:00.972411 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c978c7f-847b-43af-8741-4beb192b7339 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:00.974520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3c978c7f-847b-43af-8741-4beb192b7339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:01.100020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c978c7f-847b-43af-8741-4beb192b7339 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:01.123029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:01.123951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:01.131961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:01.134721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] PATH_INFO: `/v3/users/fcf60fa06c8a439fba19b3165396eb64` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:01.136182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf60fa06c8a439fba19b3165396eb64)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:01.139065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:01.209903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf60fa06c8a439fba19b3165396eb64'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:01.210256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf60fa06c8a439fba19b3165396eb64'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:01.223161 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf60fa06c8a439fba19b3165396eb64'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:01.225706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf60fa06c8a439fba19b3165396eb64'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:01.246130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf60fa06c8a439fba19b3165396eb64'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:01.273399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 961/1931] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:00 2022] DELETE /identity/v3/users/fcf60fa06c8a439fba19b3165396eb64 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:01.277824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad0e5-ca65-402d-8610-41196f628a72 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:01.278379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad0e5-ca65-402d-8610-41196f628a72 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:01.278836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad0e5-ca65-402d-8610-41196f628a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:01.303240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-68bad0e5-ca65-402d-8610-41196f628a72 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:01.304848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-68bad0e5-ca65-402d-8610-41196f628a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:01.453111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 971/1932] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:01.463448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:01.465224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:01.570545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb074d-f2f6-49e2-8390-0592eb8378d4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:01.575376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59261cc-8f49-420b-8f33-a9c99a082154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:01.576040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59261cc-8f49-420b-8f33-a9c99a082154 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:01.576321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59261cc-8f49-420b-8f33-a9c99a082154 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:01.576594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59261cc-8f49-420b-8f33-a9c99a082154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:01.588936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59261cc-8f49-420b-8f33-a9c99a082154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:01.600297 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59261cc-8f49-420b-8f33-a9c99a082154 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:01.608055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:01.613927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 962/1933] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:01.997733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad0e5-ca65-402d-8610-41196f628a72 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:01.999223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-68bad0e5-ca65-402d-8610-41196f628a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:02.011831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad0e5-ca65-402d-8610-41196f628a72 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:02.016720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:02.017626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:02.017913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:02.018193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] PATH_INFO: `/v3/projects/5daa8d3eaf2540918f6e0d3fbe987135` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:02.019419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5daa8d3eaf2540918f6e0d3fbe987135)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:02.021412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:02.048483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:02.076729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5daa8d3eaf2540918f6e0d3fbe987135', 'request_id': 'req-0941cd5a-59be-45d5-b6d0-ac73a0096928'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.083786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5daa8d3eaf2540918f6e0d3fbe987135', 'request_id': 'req-0941cd5a-59be-45d5-b6d0-ac73a0096928'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.091899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5daa8d3eaf2540918f6e0d3fbe987135', 'request_id': 'req-0941cd5a-59be-45d5-b6d0-ac73a0096928'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.107769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] The token cache is being invalidate because project 5daa8d3eaf2540918f6e0d3fbe987135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:02.108064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.108708 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.110317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 972/1934] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:01 2022] DELETE /identity/v3/projects/5daa8d3eaf2540918f6e0d3fbe987135 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:02.114495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59261cc-8f49-420b-8f33-a9c99a082154 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:02.124892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59261cc-8f49-420b-8f33-a9c99a082154 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:02.190996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:02.191842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:02.192138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:02.192400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] PATH_INFO: `/v3/users/d3ba3423897e42cfaf4be024eddda8c5` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:02.193688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ba3423897e42cfaf4be024eddda8c5)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:02.195759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:02.247629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ba3423897e42cfaf4be024eddda8c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.248313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ba3423897e42cfaf4be024eddda8c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.256844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ba3423897e42cfaf4be024eddda8c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.257780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ba3423897e42cfaf4be024eddda8c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.271170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ba3423897e42cfaf4be024eddda8c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.285364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 963/1935] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:02 2022] DELETE /identity/v3/users/d3ba3423897e42cfaf4be024eddda8c5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:02.751363 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:02.757553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:02.775893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941cd5a-59be-45d5-b6d0-ac73a0096928 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:02.864976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:02.865860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:02.866284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:02.866574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] PATH_INFO: `/v3/projects/91cf5f624b2e4584826f1c3d8511bd08` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:02.868008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=91cf5f624b2e4584826f1c3d8511bd08)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:02.870157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:02.967133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91cf5f624b2e4584826f1c3d8511bd08', 'request_id': 'req-42aeff7a-e659-42e8-aec5-4db9de18ea2f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.974683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91cf5f624b2e4584826f1c3d8511bd08', 'request_id': 'req-42aeff7a-e659-42e8-aec5-4db9de18ea2f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:02.987074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91cf5f624b2e4584826f1c3d8511bd08', 'request_id': 'req-42aeff7a-e659-42e8-aec5-4db9de18ea2f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.009963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] The token cache is being invalidate because project 91cf5f624b2e4584826f1c3d8511bd08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:03.010538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.011710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.013292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 973/1936] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:02 2022] DELETE /identity/v3/projects/91cf5f624b2e4584826f1c3d8511bd08 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:03.017573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:03.019016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:03.036904 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc43a44-3647-4785-a919-f81f5a0879e0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:03.103188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:03.104118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:03.104458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:03.105121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] PATH_INFO: `/v3/users/3188916bf1404f2a9bac0164bab42dd9` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:03.106036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3188916bf1404f2a9bac0164bab42dd9)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:03.108312 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:03.139888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:03.169091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3188916bf1404f2a9bac0164bab42dd9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.169866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3188916bf1404f2a9bac0164bab42dd9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.179585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3188916bf1404f2a9bac0164bab42dd9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.180568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3188916bf1404f2a9bac0164bab42dd9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.192921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d68262e-bbe9-4693-8c59-8d9d8bdc2436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3188916bf1404f2a9bac0164bab42dd9'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.207587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 964/1937] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:03 2022] DELETE /identity/v3/users/3188916bf1404f2a9bac0164bab42dd9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:03.504423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:03.510243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:03.527392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeff7a-e659-42e8-aec5-4db9de18ea2f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:03.571915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:03.593332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:03.594254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:03.594533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:03.594928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] PATH_INFO: `/v3/projects/b32c15b56c8b4f10bee920d1ea932fa5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:03.596294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32c15b56c8b4f10bee920d1ea932fa5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:03.598264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:03.649077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32c15b56c8b4f10bee920d1ea932fa5', 'request_id': 'req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.654877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32c15b56c8b4f10bee920d1ea932fa5', 'request_id': 'req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.661099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32c15b56c8b4f10bee920d1ea932fa5', 'request_id': 'req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.675740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] The token cache is being invalidate because project b32c15b56c8b4f10bee920d1ea932fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:03.676132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.676790 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f32daf94-4c50-4af2-aabc-8da6aeaa7ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:03.678359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 974/1938] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:03 2022] DELETE /identity/v3/projects/b32c15b56c8b4f10bee920d1ea932fa5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:03.684613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd531b-a7c0-46d2-8e93-4ce735e7e5e7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:03.684907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd531b-a7c0-46d2-8e93-4ce735e7e5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:03.685185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd531b-a7c0-46d2-8e93-4ce735e7e5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:03.703811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-52cd531b-a7c0-46d2-8e93-4ce735e7e5e7 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:03.705039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-52cd531b-a7c0-46d2-8e93-4ce735e7e5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:03.773748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 965/1939] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:03.782773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04377abc-a295-431e-8940-92d78cf0ad8b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:03.783064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04377abc-a295-431e-8940-92d78cf0ad8b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:03.783521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04377abc-a295-431e-8940-92d78cf0ad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:03.803039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-04377abc-a295-431e-8940-92d78cf0ad8b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:03.805192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-04377abc-a295-431e-8940-92d78cf0ad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:03.814010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 975/1940] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:03.818277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd531b-a7c0-46d2-8e93-4ce735e7e5e7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:03.830384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd531b-a7c0-46d2-8e93-4ce735e7e5e7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:03.836019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbfd0c-1a8c-4a7c-be6b-62afa0ad4209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:03.836853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbfd0c-1a8c-4a7c-be6b-62afa0ad4209 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:03.837204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbfd0c-1a8c-4a7c-be6b-62afa0ad4209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:03.837592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbfd0c-1a8c-4a7c-be6b-62afa0ad4209 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:03.838432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbfd0c-1a8c-4a7c-be6b-62afa0ad4209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:03.838863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbfd0c-1a8c-4a7c-be6b-62afa0ad4209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:03.840748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbfd0c-1a8c-4a7c-be6b-62afa0ad4209 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:03.854559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 966/1941] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:03.859209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04377abc-a295-431e-8940-92d78cf0ad8b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:03.869411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04377abc-a295-431e-8940-92d78cf0ad8b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:03.874523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9edd3-8954-4fa8-b9d8-264a91763a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:03.875280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9edd3-8954-4fa8-b9d8-264a91763a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:03.875586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9edd3-8954-4fa8-b9d8-264a91763a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:03.875854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9edd3-8954-4fa8-b9d8-264a91763a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:03.876634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9edd3-8954-4fa8-b9d8-264a91763a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:03.879340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9edd3-8954-4fa8-b9d8-264a91763a1f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:03.911001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 976/1942] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:03 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:03.920261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbfd0c-1a8c-4a7c-be6b-62afa0ad4209 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:03.949490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbfd0c-1a8c-4a7c-be6b-62afa0ad4209 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:04.018844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a2d52-0c0c-4743-95c0-2415bf70fe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:04.019873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a2d52-0c0c-4743-95c0-2415bf70fe3f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.020308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a2d52-0c0c-4743-95c0-2415bf70fe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.020744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a2d52-0c0c-4743-95c0-2415bf70fe3f admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.021489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a2d52-0c0c-4743-95c0-2415bf70fe3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:04.024419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a2d52-0c0c-4743-95c0-2415bf70fe3f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:04.051470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 967/1943] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:03 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:04.053470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9edd3-8954-4fa8-b9d8-264a91763a1f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:04.065136 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9edd3-8954-4fa8-b9d8-264a91763a1f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:04.070589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b44a18-fc23-4519-81eb-bc9327052a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:04.071034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b44a18-fc23-4519-81eb-bc9327052a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.071425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b44a18-fc23-4519-81eb-bc9327052a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.071774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b44a18-fc23-4519-81eb-bc9327052a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.072652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b44a18-fc23-4519-81eb-bc9327052a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:04.074243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b44a18-fc23-4519-81eb-bc9327052a2d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:04.087321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 977/1944] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:04.091376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a2d52-0c0c-4743-95c0-2415bf70fe3f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:04.105057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a2d52-0c0c-4743-95c0-2415bf70fe3f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:04.110111 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba17d2a-b2a9-4ec1-874c-a2b5ef93651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:04.111035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba17d2a-b2a9-4ec1-874c-a2b5ef93651f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.111422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba17d2a-b2a9-4ec1-874c-a2b5ef93651f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.111799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba17d2a-b2a9-4ec1-874c-a2b5ef93651f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.112977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba17d2a-b2a9-4ec1-874c-a2b5ef93651f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:04.114749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba17d2a-b2a9-4ec1-874c-a2b5ef93651f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:04.135438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 968/1945] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:04.139639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b44a18-fc23-4519-81eb-bc9327052a2d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:04.152951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba17d2a-b2a9-4ec1-874c-a2b5ef93651f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:04.153131 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b44a18-fc23-4519-81eb-bc9327052a2d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:04.155032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cba17d2a-b2a9-4ec1-874c-a2b5ef93651f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:04.157749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf18b5-c735-44c6-8828-c3f136b22b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:04.159532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf18b5-c735-44c6-8828-c3f136b22b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.159863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf18b5-c735-44c6-8828-c3f136b22b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.160145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf18b5-c735-44c6-8828-c3f136b22b9f admin admin] PATH_INFO: `/v3/projects/235cfa0baa33495ab83c292503eafb98/users/4970f4c1f4574ea69edfa6bb2032331b/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.162528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:04.179838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf18b5-c735-44c6-8828-c3f136b22b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=235cfa0baa33495ab83c292503eafb98, user_id=4970f4c1f4574ea69edfa6bb2032331b, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:04.182553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf18b5-c735-44c6-8828-c3f136b22b9f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:04.197974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 978/1946] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:04 2022] PUT /identity/v3/projects/235cfa0baa33495ab83c292503eafb98/users/4970f4c1f4574ea69edfa6bb2032331b/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:04.210108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea8bbc-5a2e-4860-b901-e05800597502 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.210393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea8bbc-5a2e-4860-b901-e05800597502 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.210668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea8bbc-5a2e-4860-b901-e05800597502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.222181 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba17d2a-b2a9-4ec1-874c-a2b5ef93651f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:04.240383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a6ea8bbc-5a2e-4860-b901-e05800597502 None None] MFA Rules not processed for user `4970f4c1f4574ea69edfa6bb2032331b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:04.242065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a6ea8bbc-5a2e-4860-b901-e05800597502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:04.283785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7903b2-08c1-49db-bf36-797a909b0aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:04.285299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7903b2-08c1-49db-bf36-797a909b0aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.285819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7903b2-08c1-49db-bf36-797a909b0aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.286359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7903b2-08c1-49db-bf36-797a909b0aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.289724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fd7903b2-08c1-49db-bf36-797a909b0aa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:04.309277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:04.320463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 979/1947] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:04.328015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea8bbc-5a2e-4860-b901-e05800597502 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:04.329472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a6ea8bbc-5a2e-4860-b901-e05800597502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:04.342489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea8bbc-5a2e-4860-b901-e05800597502 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:04.346245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9347a1a-8b97-487a-98f0-bf8512b87b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:04.346910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9347a1a-8b97-487a-98f0-bf8512b87b6e service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.347186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9347a1a-8b97-487a-98f0-bf8512b87b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.348303 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9347a1a-8b97-487a-98f0-bf8512b87b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.359599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9347a1a-8b97-487a-98f0-bf8512b87b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:04.361367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9347a1a-8b97-487a-98f0-bf8512b87b6e service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:04.371895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 980/1948] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:35:04.380763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7903b2-08c1-49db-bf36-797a909b0aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:04.383337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7903b2-08c1-49db-bf36-797a909b0aa6 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:04.391461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9347a1a-8b97-487a-98f0-bf8512b87b6e service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:04.393534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 969/1949] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:35:04.474704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9347a1a-8b97-487a-98f0-bf8512b87b6e service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:04.480336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:04.481219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.481505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.481770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] PATH_INFO: `/v3/users/4970f4c1f4574ea69edfa6bb2032331b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.482965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=4970f4c1f4574ea69edfa6bb2032331b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:04.485050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:04.534063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4970f4c1f4574ea69edfa6bb2032331b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:04.534749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4970f4c1f4574ea69edfa6bb2032331b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:04.542679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4970f4c1f4574ea69edfa6bb2032331b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:04.557295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4970f4c1f4574ea69edfa6bb2032331b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:04.558019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4970f4c1f4574ea69edfa6bb2032331b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:04.570705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 981/1950] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:04 2022] DELETE /identity/v3/users/4970f4c1f4574ea69edfa6bb2032331b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:04.575609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d8791-aced-4b12-90f6-c8299f63e3aa None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.576000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d8791-aced-4b12-90f6-c8299f63e3aa None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.576351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d8791-aced-4b12-90f6-c8299f63e3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.593843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c59d8791-aced-4b12-90f6-c8299f63e3aa None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:04.594982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c59d8791-aced-4b12-90f6-c8299f63e3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:04.655363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 970/1951] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:04.665378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:04.666861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:04.726487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b521d52-4c39-4182-910d-859848573dff admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:04.730811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630ba0c-7627-4669-bc72-b35ba6d00f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:04.731531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630ba0c-7627-4669-bc72-b35ba6d00f86 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:04.731836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630ba0c-7627-4669-bc72-b35ba6d00f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:04.732095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630ba0c-7627-4669-bc72-b35ba6d00f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:04.743407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630ba0c-7627-4669-bc72-b35ba6d00f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:04.744873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630ba0c-7627-4669-bc72-b35ba6d00f86 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:04.754985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 982/1952] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:05.046858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d8791-aced-4b12-90f6-c8299f63e3aa None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:05.048590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c59d8791-aced-4b12-90f6-c8299f63e3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:05.068876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d8791-aced-4b12-90f6-c8299f63e3aa None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:05.076632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:05.078635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.079093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.079591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] PATH_INFO: `/v3/projects/235cfa0baa33495ab83c292503eafb98` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.080988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=235cfa0baa33495ab83c292503eafb98)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:05.082945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:05.095831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:05.149291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '235cfa0baa33495ab83c292503eafb98', 'request_id': 'req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:05.158525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '235cfa0baa33495ab83c292503eafb98', 'request_id': 'req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:05.165816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '235cfa0baa33495ab83c292503eafb98', 'request_id': 'req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:05.177469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] The token cache is being invalidate because project 235cfa0baa33495ab83c292503eafb98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:05.177759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:05.178508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c22fc0fe-b86e-40ce-9f38-5708d8ef49c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:05.180176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 971/1953] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:05 2022] DELETE /identity/v3/projects/235cfa0baa33495ab83c292503eafb98 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:05.185151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5443f94-e470-48fa-9630-1c0ce2b05127 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.185614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5443f94-e470-48fa-9630-1c0ce2b05127 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.185725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5443f94-e470-48fa-9630-1c0ce2b05127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.211683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:05.214233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e5443f94-e470-48fa-9630-1c0ce2b05127 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:05.216284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e5443f94-e470-48fa-9630-1c0ce2b05127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:05.278898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 983/1954] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:05.288737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953f694-a369-41b1-a119-b525a24f8241 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.289361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953f694-a369-41b1-a119-b525a24f8241 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.289361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953f694-a369-41b1-a119-b525a24f8241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.310176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8953f694-a369-41b1-a119-b525a24f8241 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:05.311479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8953f694-a369-41b1-a119-b525a24f8241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:05.320121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 972/1955] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:05.324513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5443f94-e470-48fa-9630-1c0ce2b05127 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:05.335833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5443f94-e470-48fa-9630-1c0ce2b05127 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:05.341559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b8288-d68c-4078-a8ba-fa68ac4993dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:05.342229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b8288-d68c-4078-a8ba-fa68ac4993dc admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.342538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b8288-d68c-4078-a8ba-fa68ac4993dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.342795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b8288-d68c-4078-a8ba-fa68ac4993dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.343581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b8288-d68c-4078-a8ba-fa68ac4993dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:05.343887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b8288-d68c-4078-a8ba-fa68ac4993dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:05.345550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b8288-d68c-4078-a8ba-fa68ac4993dc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:05.358259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 984/1956] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:05.362890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953f694-a369-41b1-a119-b525a24f8241 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:05.373374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953f694-a369-41b1-a119-b525a24f8241 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:05.379334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71eeea1-296d-417c-8097-0758a8e76f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:05.380025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71eeea1-296d-417c-8097-0758a8e76f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.380300 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71eeea1-296d-417c-8097-0758a8e76f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.380561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71eeea1-296d-417c-8097-0758a8e76f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.381254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71eeea1-296d-417c-8097-0758a8e76f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:05.383238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71eeea1-296d-417c-8097-0758a8e76f91 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:05.413231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 973/1957] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:05 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:05.418162 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b8288-d68c-4078-a8ba-fa68ac4993dc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:05.432439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b8288-d68c-4078-a8ba-fa68ac4993dc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:05.492333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd67d5-2db2-47ae-976a-13f1db192b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:05.493241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd67d5-2db2-47ae-976a-13f1db192b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.493531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd67d5-2db2-47ae-976a-13f1db192b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.493811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd67d5-2db2-47ae-976a-13f1db192b03 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.494506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd67d5-2db2-47ae-976a-13f1db192b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:05.496480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd67d5-2db2-47ae-976a-13f1db192b03 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:05.522975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 985/1958] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:05 2022] POST /identity/v3/users => generated 281 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:05.524278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71eeea1-296d-417c-8097-0758a8e76f91 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:05.535243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71eeea1-296d-417c-8097-0758a8e76f91 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:05.540436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5f04b-174e-4828-a98c-7bc8278ba72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:05.541103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5f04b-174e-4828-a98c-7bc8278ba72d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.541385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5f04b-174e-4828-a98c-7bc8278ba72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.541653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5f04b-174e-4828-a98c-7bc8278ba72d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.542415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5f04b-174e-4828-a98c-7bc8278ba72d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:05.544053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5f04b-174e-4828-a98c-7bc8278ba72d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:05.561811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 974/1959] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:05.563119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd67d5-2db2-47ae-976a-13f1db192b03 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:05.574562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd67d5-2db2-47ae-976a-13f1db192b03 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:05.579710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce14831-7d06-49c4-aa40-8148e9a924d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:05.580389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce14831-7d06-49c4-aa40-8148e9a924d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.580672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce14831-7d06-49c4-aa40-8148e9a924d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.580974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce14831-7d06-49c4-aa40-8148e9a924d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.582056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce14831-7d06-49c4-aa40-8148e9a924d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:05.583415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce14831-7d06-49c4-aa40-8148e9a924d3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:05.594213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 986/1960] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:05.598341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5f04b-174e-4828-a98c-7bc8278ba72d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:05.608846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5f04b-174e-4828-a98c-7bc8278ba72d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:05.613701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393392a5-15b2-42c3-a46d-fbd08cc76c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:05.614612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393392a5-15b2-42c3-a46d-fbd08cc76c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.614902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393392a5-15b2-42c3-a46d-fbd08cc76c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.615183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393392a5-15b2-42c3-a46d-fbd08cc76c55 admin admin] PATH_INFO: `/v3/projects/2bab74b3847740b19382ad8503370719/users/dd777fb05f4e4abdadc20430e00fb7be/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.630315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393392a5-15b2-42c3-a46d-fbd08cc76c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bab74b3847740b19382ad8503370719, user_id=dd777fb05f4e4abdadc20430e00fb7be, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:05.633294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393392a5-15b2-42c3-a46d-fbd08cc76c55 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:05.649396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 975/1961] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:05 2022] PUT /identity/v3/projects/2bab74b3847740b19382ad8503370719/users/dd777fb05f4e4abdadc20430e00fb7be/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:05.663744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d3809-380f-4d33-a22d-0191bdfcd81e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.664030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d3809-380f-4d33-a22d-0191bdfcd81e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.664367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d3809-380f-4d33-a22d-0191bdfcd81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.694409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2e5d3809-380f-4d33-a22d-0191bdfcd81e None None] MFA Rules not processed for user `dd777fb05f4e4abdadc20430e00fb7be`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:05.695773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2e5d3809-380f-4d33-a22d-0191bdfcd81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:05.766379 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 987/1962] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:05.802708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393392a5-15b2-42c3-a46d-fbd08cc76c55 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:05.804542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-393392a5-15b2-42c3-a46d-fbd08cc76c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:05.865453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393392a5-15b2-42c3-a46d-fbd08cc76c55 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:05.869457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e57c15-5786-4e2a-b29d-8d75f05314d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:05.870113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e57c15-5786-4e2a-b29d-8d75f05314d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.870392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e57c15-5786-4e2a-b29d-8d75f05314d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.870661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e57c15-5786-4e2a-b29d-8d75f05314d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.879708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e57c15-5786-4e2a-b29d-8d75f05314d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:05.881184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e57c15-5786-4e2a-b29d-8d75f05314d3 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:05.897868 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 976/1963] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:05.905330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d3809-380f-4d33-a22d-0191bdfcd81e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:05.909546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:05.972773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d3809-380f-4d33-a22d-0191bdfcd81e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:05.977568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:05.978487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:05.978772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:05.979055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] PATH_INFO: `/v3/users/dd777fb05f4e4abdadc20430e00fb7be` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:05.980440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd777fb05f4e4abdadc20430e00fb7be)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:05.982633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:06.039561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd777fb05f4e4abdadc20430e00fb7be'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.040279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd777fb05f4e4abdadc20430e00fb7be'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.049390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd777fb05f4e4abdadc20430e00fb7be'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.064100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd777fb05f4e4abdadc20430e00fb7be'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.065266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd777fb05f4e4abdadc20430e00fb7be'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.079226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 988/1964] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:05 2022] DELETE /identity/v3/users/dd777fb05f4e4abdadc20430e00fb7be => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:06.084134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1363e-7253-4e21-bd43-bfe0d90d2458 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:06.084436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1363e-7253-4e21-bd43-bfe0d90d2458 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:06.084795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1363e-7253-4e21-bd43-bfe0d90d2458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:06.103615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e3b1363e-7253-4e21-bd43-bfe0d90d2458 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:06.104954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e3b1363e-7253-4e21-bd43-bfe0d90d2458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:06.150444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:06.183033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 977/1965] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:06.193406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:06.195322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:06.276487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb42e17-9b7a-4fee-92d0-81df8482b945 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:06.280360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2804501-dd4f-447f-ae58-e7a122b7f90c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:06.281029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804501-dd4f-447f-ae58-e7a122b7f90c service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:06.281315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804501-dd4f-447f-ae58-e7a122b7f90c service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:06.281593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2804501-dd4f-447f-ae58-e7a122b7f90c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:06.291261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2804501-dd4f-447f-ae58-e7a122b7f90c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:06.292866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2804501-dd4f-447f-ae58-e7a122b7f90c service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:06.303600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 989/1966] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:06.592356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1363e-7253-4e21-bd43-bfe0d90d2458 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:06.593965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e3b1363e-7253-4e21-bd43-bfe0d90d2458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:06.632317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1363e-7253-4e21-bd43-bfe0d90d2458 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:06.637976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:06.639153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:06.639329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:06.639617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] PATH_INFO: `/v3/projects/2bab74b3847740b19382ad8503370719` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:06.640885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bab74b3847740b19382ad8503370719)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:06.642831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:06.693726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bab74b3847740b19382ad8503370719', 'request_id': 'req-d5928e1e-717f-4972-b45c-7f7d53427f0a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.702496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bab74b3847740b19382ad8503370719', 'request_id': 'req-d5928e1e-717f-4972-b45c-7f7d53427f0a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.709217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bab74b3847740b19382ad8503370719', 'request_id': 'req-d5928e1e-717f-4972-b45c-7f7d53427f0a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.720077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] The token cache is being invalidate because project 2bab74b3847740b19382ad8503370719 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:06.720401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.721073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d5928e1e-717f-4972-b45c-7f7d53427f0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:06.722628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 978/1967] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:06 2022] DELETE /identity/v3/projects/2bab74b3847740b19382ad8503370719 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:06.728014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ef37-4f1a-4464-b9ab-64191720880d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:06.728305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ef37-4f1a-4464-b9ab-64191720880d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:06.728625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4ef37-4f1a-4464-b9ab-64191720880d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:06.746362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-75e4ef37-4f1a-4464-b9ab-64191720880d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:06.747610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-75e4ef37-4f1a-4464-b9ab-64191720880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:06.767776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:06.810680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 990/1968] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:06.819769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c70e6b-0bcf-40db-b55d-9f508b78f582 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:06.820089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c70e6b-0bcf-40db-b55d-9f508b78f582 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:06.820368 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c70e6b-0bcf-40db-b55d-9f508b78f582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:06.840346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-72c70e6b-0bcf-40db-b55d-9f508b78f582 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:06.841560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-72c70e6b-0bcf-40db-b55d-9f508b78f582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:06.850579 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 979/1969] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:06.855132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4ef37-4f1a-4464-b9ab-64191720880d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:06.866454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4ef37-4f1a-4464-b9ab-64191720880d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:06.872736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1eb5-974e-4d30-8268-898bc7018a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:06.873440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1eb5-974e-4d30-8268-898bc7018a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:06.873713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1eb5-974e-4d30-8268-898bc7018a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:06.873989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1eb5-974e-4d30-8268-898bc7018a80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:06.874720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee1eb5-974e-4d30-8268-898bc7018a80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:06.875023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee1eb5-974e-4d30-8268-898bc7018a80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:06.876662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee1eb5-974e-4d30-8268-898bc7018a80 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:06.889240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 991/1970] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:06.893779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c70e6b-0bcf-40db-b55d-9f508b78f582 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:06.903793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c70e6b-0bcf-40db-b55d-9f508b78f582 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:06.910224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ddd01-442a-48b5-ac1d-170169d402ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:06.910886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ddd01-442a-48b5-ac1d-170169d402ac admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:06.911162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ddd01-442a-48b5-ac1d-170169d402ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:06.911470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ddd01-442a-48b5-ac1d-170169d402ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:06.912192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ddd01-442a-48b5-ac1d-170169d402ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:06.914264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ddd01-442a-48b5-ac1d-170169d402ac admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:06.946152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 980/1971] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:06 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:06.947453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1eb5-974e-4d30-8268-898bc7018a80 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:06.959671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1eb5-974e-4d30-8268-898bc7018a80 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:07.024745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918dbda1-3dbd-4bea-8170-7fdfbf2d6eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:07.025495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918dbda1-3dbd-4bea-8170-7fdfbf2d6eef admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:07.025785 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918dbda1-3dbd-4bea-8170-7fdfbf2d6eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:07.026067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918dbda1-3dbd-4bea-8170-7fdfbf2d6eef admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:07.026874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918dbda1-3dbd-4bea-8170-7fdfbf2d6eef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:07.028982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918dbda1-3dbd-4bea-8170-7fdfbf2d6eef admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:07.062027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 992/1972] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:06 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:07.064003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ddd01-442a-48b5-ac1d-170169d402ac admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:07.076507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ddd01-442a-48b5-ac1d-170169d402ac admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:07.085641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8be9f-482b-4a16-beff-f14cc3326879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:07.086403 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8be9f-482b-4a16-beff-f14cc3326879 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:07.086779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8be9f-482b-4a16-beff-f14cc3326879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:07.087128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8be9f-482b-4a16-beff-f14cc3326879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:07.088097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8be9f-482b-4a16-beff-f14cc3326879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:07.089789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8be9f-482b-4a16-beff-f14cc3326879 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:07.101441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 981/1973] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:07.105686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918dbda1-3dbd-4bea-8170-7fdfbf2d6eef admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:07.116350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918dbda1-3dbd-4bea-8170-7fdfbf2d6eef admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:07.121066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9047c0-cc54-42f2-b04f-83b90cb18598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:07.121795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9047c0-cc54-42f2-b04f-83b90cb18598 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:07.122074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9047c0-cc54-42f2-b04f-83b90cb18598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:07.122342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9047c0-cc54-42f2-b04f-83b90cb18598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:07.123080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9047c0-cc54-42f2-b04f-83b90cb18598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:07.124793 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9047c0-cc54-42f2-b04f-83b90cb18598 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:07.138531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 993/1974] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:07.142056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8be9f-482b-4a16-beff-f14cc3326879 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:07.152378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8be9f-482b-4a16-beff-f14cc3326879 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:07.157177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec369c-ddf1-4ddb-9398-d293c6e5d58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:07.158087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec369c-ddf1-4ddb-9398-d293c6e5d58e admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:07.158496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec369c-ddf1-4ddb-9398-d293c6e5d58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:07.158773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec369c-ddf1-4ddb-9398-d293c6e5d58e admin admin] PATH_INFO: `/v3/projects/e59a8c977471441e8d569ab1ba8f472b/users/e945141497a9424e910b0d4c77dd32e7/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:07.174685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec369c-ddf1-4ddb-9398-d293c6e5d58e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e59a8c977471441e8d569ab1ba8f472b, user_id=e945141497a9424e910b0d4c77dd32e7, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:07.177465 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec369c-ddf1-4ddb-9398-d293c6e5d58e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:07.191719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 982/1975] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:07 2022] PUT /identity/v3/projects/e59a8c977471441e8d569ab1ba8f472b/users/e945141497a9424e910b0d4c77dd32e7/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:07.202039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25005890-7581-4df4-b2de-b73a2472e813 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:07.202538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25005890-7581-4df4-b2de-b73a2472e813 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:07.202647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25005890-7581-4df4-b2de-b73a2472e813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:07.230000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-25005890-7581-4df4-b2de-b73a2472e813 None None] MFA Rules not processed for user `e945141497a9424e910b0d4c77dd32e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:07.231207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-25005890-7581-4df4-b2de-b73a2472e813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:07.320081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 994/1976] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:07.333701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec369c-ddf1-4ddb-9398-d293c6e5d58e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:07.336110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4cec369c-ddf1-4ddb-9398-d293c6e5d58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:07.402421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec369c-ddf1-4ddb-9398-d293c6e5d58e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:07.407720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55187ad2-e655-41f5-bfed-0adf8dee7464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:07.408410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55187ad2-e655-41f5-bfed-0adf8dee7464 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:07.409317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55187ad2-e655-41f5-bfed-0adf8dee7464 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:07.409606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55187ad2-e655-41f5-bfed-0adf8dee7464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:07.420825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55187ad2-e655-41f5-bfed-0adf8dee7464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:07.422154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55187ad2-e655-41f5-bfed-0adf8dee7464 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:07.431059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:07.450645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 983/1977] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:35:07.920203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92307a-cd6e-474d-b789-a1c4c3ef8cd7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:07.920686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92307a-cd6e-474d-b789-a1c4c3ef8cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:07.921778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92307a-cd6e-474d-b789-a1c4c3ef8cd7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:07.923675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 995/1978] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:35:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:35:07.933761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9f00-e4fb-4cbd-b91a-76d21eab12a2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:07.937109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9f00-e4fb-4cbd-b91a-76d21eab12a2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:07.937969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9f00-e4fb-4cbd-b91a-76d21eab12a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:08.006584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a21e9f00-e4fb-4cbd-b91a-76d21eab12a2 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:08.011072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a21e9f00-e4fb-4cbd-b91a-76d21eab12a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:08.215266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 984/1979] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:35:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:08.223572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92307a-cd6e-474d-b789-a1c4c3ef8cd7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:08.226511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0f92307a-cd6e-474d-b789-a1c4c3ef8cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:08.260438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92307a-cd6e-474d-b789-a1c4c3ef8cd7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:08.266322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0410ef-ecab-4e30-8c7d-60c271c88ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:08.267072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0410ef-ecab-4e30-8c7d-60c271c88ded service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:08.267439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0410ef-ecab-4e30-8c7d-60c271c88ded service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:08.267779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0410ef-ecab-4e30-8c7d-60c271c88ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:08.281791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0410ef-ecab-4e30-8c7d-60c271c88ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:08.283765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0410ef-ecab-4e30-8c7d-60c271c88ded service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:08.296430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 996/1980] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:08.767857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9f00-e4fb-4cbd-b91a-76d21eab12a2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:08.769374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a21e9f00-e4fb-4cbd-b91a-76d21eab12a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:08.847205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9f00-e4fb-4cbd-b91a-76d21eab12a2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:08.852268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:08.853167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:08.853453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:08.853866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] PATH_INFO: `/v3/users/aef8f0a7bacc4f6b9bb6ccf8e4d86ecd` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:08.855524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aef8f0a7bacc4f6b9bb6ccf8e4d86ecd)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:08.857593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:08.928941 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aef8f0a7bacc4f6b9bb6ccf8e4d86ecd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:08.930902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aef8f0a7bacc4f6b9bb6ccf8e4d86ecd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:08.944723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aef8f0a7bacc4f6b9bb6ccf8e4d86ecd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:08.946925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aef8f0a7bacc4f6b9bb6ccf8e4d86ecd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:08.964410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aef8f0a7bacc4f6b9bb6ccf8e4d86ecd'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:08.981835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 985/1981] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:08 2022] DELETE /identity/v3/users/aef8f0a7bacc4f6b9bb6ccf8e4d86ecd => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:08.986160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c8b73-8e4f-4908-9c1b-c4456d245180 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:08.986478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c8b73-8e4f-4908-9c1b-c4456d245180 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:08.986771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c8b73-8e4f-4908-9c1b-c4456d245180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:09.000002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:09.007609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e55c8b73-8e4f-4908-9c1b-c4456d245180 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:09.009277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e55c8b73-8e4f-4908-9c1b-c4456d245180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:09.085409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 997/1982] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:09.095015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:09.096675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:09.112597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:09.163363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21271e9b-dede-4195-8e3c-512c0a4c4935 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:09.168299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3359e6-60cf-4e4c-9dbe-31701cbd65cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:09.169517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3359e6-60cf-4e4c-9dbe-31701cbd65cf service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:09.169860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3359e6-60cf-4e4c-9dbe-31701cbd65cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:09.170470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3359e6-60cf-4e4c-9dbe-31701cbd65cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:09.182252 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3359e6-60cf-4e4c-9dbe-31701cbd65cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:09.183972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3359e6-60cf-4e4c-9dbe-31701cbd65cf service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:09.195693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 986/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:09.686679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c8b73-8e4f-4908-9c1b-c4456d245180 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:09.688305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e55c8b73-8e4f-4908-9c1b-c4456d245180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:09.701179 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c8b73-8e4f-4908-9c1b-c4456d245180 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:09.706948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:09.707894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:09.708176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:09.708452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] PATH_INFO: `/v3/projects/35dbd2c8211e4ccf95607e3f6a329294` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:09.709862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=35dbd2c8211e4ccf95607e3f6a329294)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:09.711942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:09.763047 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35dbd2c8211e4ccf95607e3f6a329294', 'request_id': 'req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.769483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35dbd2c8211e4ccf95607e3f6a329294', 'request_id': 'req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.776494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35dbd2c8211e4ccf95607e3f6a329294', 'request_id': 'req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.789712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] The token cache is being invalidate because project 35dbd2c8211e4ccf95607e3f6a329294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:09.790005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.790622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.792223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 998/1984] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:09 2022] DELETE /identity/v3/projects/35dbd2c8211e4ccf95607e3f6a329294 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:09.797074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3359e6-60cf-4e4c-9dbe-31701cbd65cf service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:09.807531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3359e6-60cf-4e4c-9dbe-31701cbd65cf service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:09.868817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:09.869721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:09.870007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:09.870284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] PATH_INFO: `/v3/users/b598bb94abd24e29b2e56e897505e7ec` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:09.871587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b598bb94abd24e29b2e56e897505e7ec)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:09.873576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:09.928764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b598bb94abd24e29b2e56e897505e7ec'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.929414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b598bb94abd24e29b2e56e897505e7ec'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.938311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b598bb94abd24e29b2e56e897505e7ec'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.938903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b598bb94abd24e29b2e56e897505e7ec'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.953649 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b598bb94abd24e29b2e56e897505e7ec'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:09.965783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 987/1985] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:09 2022] DELETE /identity/v3/users/b598bb94abd24e29b2e56e897505e7ec => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:10.388204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:10.389773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:10.404034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cdfa3-09c3-4d7d-92d9-ce336403a69a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:10.464876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:10.465757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:10.466032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:10.466304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] PATH_INFO: `/v3/projects/5b36792942584f5289876f74e9619452` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:10.467597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b36792942584f5289876f74e9619452)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:10.469524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:10.518469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:10.545683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b36792942584f5289876f74e9619452', 'request_id': 'req-d8067bc0-6284-44a4-8674-abf29216bbf9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.552321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b36792942584f5289876f74e9619452', 'request_id': 'req-d8067bc0-6284-44a4-8674-abf29216bbf9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.561149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b36792942584f5289876f74e9619452', 'request_id': 'req-d8067bc0-6284-44a4-8674-abf29216bbf9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.580131 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] The token cache is being invalidate because project 5b36792942584f5289876f74e9619452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:10.580458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.581369 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.587115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 999/1986] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:10 2022] DELETE /identity/v3/projects/5b36792942584f5289876f74e9619452 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:10.588711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:10.593103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:10.609350 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90877b5d-9401-4bb4-82e1-ac4c849de420 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:10.645956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:10.675442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:10.676362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:10.676644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:10.676918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] PATH_INFO: `/v3/users/1513689f654e4cba8316819091af23ca` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:10.678877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1513689f654e4cba8316819091af23ca)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:10.680903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:10.738585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1513689f654e4cba8316819091af23ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.739255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1513689f654e4cba8316819091af23ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.748934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1513689f654e4cba8316819091af23ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.749563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1513689f654e4cba8316819091af23ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.762426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-117d5f25-d00d-4ffa-9939-9771ade3da72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1513689f654e4cba8316819091af23ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:10.776082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 988/1987] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:10 2022] DELETE /identity/v3/users/1513689f654e4cba8316819091af23ca => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:11.064519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:11.066095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:11.079723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8067bc0-6284-44a4-8674-abf29216bbf9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:11.151406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:11.152319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.152602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.152878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] PATH_INFO: `/v3/projects/709ebb39f1084a27b66caa7ec1546d8c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.154078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=709ebb39f1084a27b66caa7ec1546d8c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:11.156077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:11.207178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '709ebb39f1084a27b66caa7ec1546d8c', 'request_id': 'req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:11.213756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '709ebb39f1084a27b66caa7ec1546d8c', 'request_id': 'req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:11.223912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '709ebb39f1084a27b66caa7ec1546d8c', 'request_id': 'req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:11.239276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] The token cache is being invalidate because project 709ebb39f1084a27b66caa7ec1546d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:11.239585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:11.240235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db5fdaf-04e1-4dfa-a241-9206dbc5feaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:11.241870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1000/1988] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:11 2022] DELETE /identity/v3/projects/709ebb39f1084a27b66caa7ec1546d8c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:11.248299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d6748-b4a6-412a-9bb2-c30e548f7a06 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.248754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d6748-b4a6-412a-9bb2-c30e548f7a06 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.248840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d6748-b4a6-412a-9bb2-c30e548f7a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.268545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-329d6748-b4a6-412a-9bb2-c30e548f7a06 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:11.269711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-329d6748-b4a6-412a-9bb2-c30e548f7a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:11.341801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 989/1989] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:11.351323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082acda6-ec37-47a3-9ce2-747f6f16ddd4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.351712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082acda6-ec37-47a3-9ce2-747f6f16ddd4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.352089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082acda6-ec37-47a3-9ce2-747f6f16ddd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.372853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-082acda6-ec37-47a3-9ce2-747f6f16ddd4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:11.374095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-082acda6-ec37-47a3-9ce2-747f6f16ddd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:11.384072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1001/1990] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:11.388009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d6748-b4a6-412a-9bb2-c30e548f7a06 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:11.398699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d6748-b4a6-412a-9bb2-c30e548f7a06 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:11.403236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee048f-3221-471b-a824-8bf1987f56a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:11.403960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee048f-3221-471b-a824-8bf1987f56a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.404239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee048f-3221-471b-a824-8bf1987f56a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.404513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee048f-3221-471b-a824-8bf1987f56a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.405262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee048f-3221-471b-a824-8bf1987f56a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:11.405567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee048f-3221-471b-a824-8bf1987f56a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:11.407190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee048f-3221-471b-a824-8bf1987f56a4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:11.422917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 990/1991] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:35:11.424177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082acda6-ec37-47a3-9ce2-747f6f16ddd4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:11.435048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082acda6-ec37-47a3-9ce2-747f6f16ddd4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:11.439804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2595c-cf0f-4715-9341-d68ec9736d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:11.440475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2595c-cf0f-4715-9341-d68ec9736d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.440765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2595c-cf0f-4715-9341-d68ec9736d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.441071 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2595c-cf0f-4715-9341-d68ec9736d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.441766 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2595c-cf0f-4715-9341-d68ec9736d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:11.443718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2595c-cf0f-4715-9341-d68ec9736d74 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:11.475864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1002/1992] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:11 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:11.477061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee048f-3221-471b-a824-8bf1987f56a4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:11.487804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee048f-3221-471b-a824-8bf1987f56a4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:11.533209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:11.559395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6629e-4f51-426c-b759-95b57119489c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:11.560172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6629e-4f51-426c-b759-95b57119489c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.560537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6629e-4f51-426c-b759-95b57119489c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.560861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6629e-4f51-426c-b759-95b57119489c admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.561702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6629e-4f51-426c-b759-95b57119489c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:11.563795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6629e-4f51-426c-b759-95b57119489c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:11.585110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 991/1993] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:11 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:11.589850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2595c-cf0f-4715-9341-d68ec9736d74 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:11.605885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2595c-cf0f-4715-9341-d68ec9736d74 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:11.612962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145951e5-824a-4e58-8ed7-5a3c32ed2bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:11.613649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145951e5-824a-4e58-8ed7-5a3c32ed2bff admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.613930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145951e5-824a-4e58-8ed7-5a3c32ed2bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.614269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145951e5-824a-4e58-8ed7-5a3c32ed2bff admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.614995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145951e5-824a-4e58-8ed7-5a3c32ed2bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:11.616982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145951e5-824a-4e58-8ed7-5a3c32ed2bff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:11.628188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1003/1994] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:11.632680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6629e-4f51-426c-b759-95b57119489c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:11.642942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6629e-4f51-426c-b759-95b57119489c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:11.648115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc3368-e8d8-4339-aaf2-bb24938df87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:11.648796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc3368-e8d8-4339-aaf2-bb24938df87f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.649080 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc3368-e8d8-4339-aaf2-bb24938df87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.649356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc3368-e8d8-4339-aaf2-bb24938df87f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.650082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc3368-e8d8-4339-aaf2-bb24938df87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:11.651752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc3368-e8d8-4339-aaf2-bb24938df87f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:11.665554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 992/1995] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:11.666822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145951e5-824a-4e58-8ed7-5a3c32ed2bff admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:11.677912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145951e5-824a-4e58-8ed7-5a3c32ed2bff admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:11.683521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8483877-7e94-4048-b61f-f3d60ca2cbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:11.684407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8483877-7e94-4048-b61f-f3d60ca2cbf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.684688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8483877-7e94-4048-b61f-f3d60ca2cbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.684963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8483877-7e94-4048-b61f-f3d60ca2cbf1 admin admin] PATH_INFO: `/v3/projects/36cc35de74da4061b0388fff19af3af2/users/32a418fa5623468e85e13f24be2738ca/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.693182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:11.703034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8483877-7e94-4048-b61f-f3d60ca2cbf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cc35de74da4061b0388fff19af3af2, user_id=32a418fa5623468e85e13f24be2738ca, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:11.706590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8483877-7e94-4048-b61f-f3d60ca2cbf1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:11.721457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1004/1996] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:11 2022] PUT /identity/v3/projects/36cc35de74da4061b0388fff19af3af2/users/32a418fa5623468e85e13f24be2738ca/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:11.734801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12dd86-863c-4763-bd9f-39776e9ed2bd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.735092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12dd86-863c-4763-bd9f-39776e9ed2bd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.735373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12dd86-863c-4763-bd9f-39776e9ed2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.764292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5e12dd86-863c-4763-bd9f-39776e9ed2bd None None] MFA Rules not processed for user `32a418fa5623468e85e13f24be2738ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:11.766086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5e12dd86-863c-4763-bd9f-39776e9ed2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:11.841373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 993/1997] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:11.846046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8483877-7e94-4048-b61f-f3d60ca2cbf1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:11.912238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8483877-7e94-4048-b61f-f3d60ca2cbf1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:11.917187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdede165-9150-430e-88f5-ce0a01cd2c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:11.917870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdede165-9150-430e-88f5-ce0a01cd2c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:11.918156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdede165-9150-430e-88f5-ce0a01cd2c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:11.918441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdede165-9150-430e-88f5-ce0a01cd2c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:11.919135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdede165-9150-430e-88f5-ce0a01cd2c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:11.921474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdede165-9150-430e-88f5-ce0a01cd2c8d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:11.952722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1005/1998] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:11 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:11.953965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12dd86-863c-4763-bd9f-39776e9ed2bd None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:11.965628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12dd86-863c-4763-bd9f-39776e9ed2bd None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:12.025233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf45d1-bc59-4f91-a60e-e619cc838015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:12.025820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf45d1-bc59-4f91-a60e-e619cc838015 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:12.026103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf45d1-bc59-4f91-a60e-e619cc838015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:12.026375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf45d1-bc59-4f91-a60e-e619cc838015 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:12.027069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf45d1-bc59-4f91-a60e-e619cc838015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:12.029067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf45d1-bc59-4f91-a60e-e619cc838015 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:12.053262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 994/1999] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:11 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:12.057580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdede165-9150-430e-88f5-ce0a01cd2c8d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:12.069362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdede165-9150-430e-88f5-ce0a01cd2c8d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:12.075413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d9457-225b-4374-befe-b1f03c54ea9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:12.076222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d9457-225b-4374-befe-b1f03c54ea9f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:12.076587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d9457-225b-4374-befe-b1f03c54ea9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:12.076885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d9457-225b-4374-befe-b1f03c54ea9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:12.077621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d9457-225b-4374-befe-b1f03c54ea9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:12.079276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d9457-225b-4374-befe-b1f03c54ea9f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:12.090445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1006/2000] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:12.094828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf45d1-bc59-4f91-a60e-e619cc838015 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:12.106109 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf45d1-bc59-4f91-a60e-e619cc838015 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:12.111810 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcebb3f-d228-4b7f-bf2b-db36b88a375a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:12.112761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcebb3f-d228-4b7f-bf2b-db36b88a375a admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:12.113049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcebb3f-d228-4b7f-bf2b-db36b88a375a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:12.113303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcebb3f-d228-4b7f-bf2b-db36b88a375a admin admin] PATH_INFO: `/v3/projects/55ca3bd2637e4fb5b74f7c0da43e3e7a/users/6b922ba2351d46b5a7301f198952cf6e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:12.130217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcebb3f-d228-4b7f-bf2b-db36b88a375a admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ca3bd2637e4fb5b74f7c0da43e3e7a, user_id=6b922ba2351d46b5a7301f198952cf6e, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:12.132949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcebb3f-d228-4b7f-bf2b-db36b88a375a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:12.147462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 995/2001] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:12 2022] PUT /identity/v3/projects/55ca3bd2637e4fb5b74f7c0da43e3e7a/users/6b922ba2351d46b5a7301f198952cf6e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:12.151688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d9457-225b-4374-befe-b1f03c54ea9f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:12.162123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d9457-225b-4374-befe-b1f03c54ea9f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:12.218286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf61b62-76b4-450c-9f96-9f0e41d9cdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:12.218986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61b62-76b4-450c-9f96-9f0e41d9cdaf admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:12.219404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61b62-76b4-450c-9f96-9f0e41d9cdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:12.219735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61b62-76b4-450c-9f96-9f0e41d9cdaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:12.220325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf61b62-76b4-450c-9f96-9f0e41d9cdaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:12.222010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf61b62-76b4-450c-9f96-9f0e41d9cdaf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:12.231893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1007/2002] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:12.236533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcebb3f-d228-4b7f-bf2b-db36b88a375a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:12.247838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcebb3f-d228-4b7f-bf2b-db36b88a375a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:12.253115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdc60d-284f-404c-9700-b08fb90fed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:12.253986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdc60d-284f-404c-9700-b08fb90fed95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:12.254269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdc60d-284f-404c-9700-b08fb90fed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:12.254523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdc60d-284f-404c-9700-b08fb90fed95 admin admin] PATH_INFO: `/v3/projects/55ca3bd2637e4fb5b74f7c0da43e3e7a/users/6b922ba2351d46b5a7301f198952cf6e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:12.256851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdc60d-284f-404c-9700-b08fb90fed95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55ca3bd2637e4fb5b74f7c0da43e3e7a, user_id=6b922ba2351d46b5a7301f198952cf6e, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:12.259506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdc60d-284f-404c-9700-b08fb90fed95 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:12.266366 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:12.274123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 996/2003] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:12 2022] PUT /identity/v3/projects/55ca3bd2637e4fb5b74f7c0da43e3e7a/users/6b922ba2351d46b5a7301f198952cf6e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:12.284506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9fa69-b16c-4f65-a745-79434964580a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:12.284736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9fa69-b16c-4f65-a745-79434964580a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:12.285018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9fa69-b16c-4f65-a745-79434964580a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:12.312886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dec9fa69-b16c-4f65-a745-79434964580a None None] MFA Rules not processed for user `6b922ba2351d46b5a7301f198952cf6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:12.314097 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dec9fa69-b16c-4f65-a745-79434964580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:12.358341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:12.385485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1008/2004] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:12.399744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdc60d-284f-404c-9700-b08fb90fed95 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:12.401852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fafdc60d-284f-404c-9700-b08fb90fed95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:12.464541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdc60d-284f-404c-9700-b08fb90fed95 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:12.468886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca17f-3121-44a0-a23b-80c34f2bdc3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:12.469525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca17f-3121-44a0-a23b-80c34f2bdc3b service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:12.469804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca17f-3121-44a0-a23b-80c34f2bdc3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:12.470089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca17f-3121-44a0-a23b-80c34f2bdc3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:12.480765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ca17f-3121-44a0-a23b-80c34f2bdc3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:12.482206 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ca17f-3121-44a0-a23b-80c34f2bdc3b service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:12.492194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 997/2005] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:12.581326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9fa69-b16c-4f65-a745-79434964580a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:12.592291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9fa69-b16c-4f65-a745-79434964580a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:12.597329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03500e-33eb-4b7e-a8d5-2c7395d06d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:12.598067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03500e-33eb-4b7e-a8d5-2c7395d06d3f service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:12.598253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03500e-33eb-4b7e-a8d5-2c7395d06d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:12.598524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03500e-33eb-4b7e-a8d5-2c7395d06d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:12.671784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03500e-33eb-4b7e-a8d5-2c7395d06d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:12.673362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc03500e-33eb-4b7e-a8d5-2c7395d06d3f service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:12.683921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1009/2006] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:12.989988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca17f-3121-44a0-a23b-80c34f2bdc3b service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:13.059191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca17f-3121-44a0-a23b-80c34f2bdc3b service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:13.063978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:13.064938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:13.065280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:13.065625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] PATH_INFO: `/v3/users/32a418fa5623468e85e13f24be2738ca` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:13.066879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a418fa5623468e85e13f24be2738ca)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:13.069023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:13.124890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a418fa5623468e85e13f24be2738ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.129698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a418fa5623468e85e13f24be2738ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.136907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a418fa5623468e85e13f24be2738ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.138046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a418fa5623468e85e13f24be2738ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.151568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a418fa5623468e85e13f24be2738ca'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.167343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 998/2007] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:12 2022] DELETE /identity/v3/users/32a418fa5623468e85e13f24be2738ca => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:13.172820 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb59234-97ed-4cea-849e-bee0485bdd23 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:13.173404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb59234-97ed-4cea-849e-bee0485bdd23 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:13.173690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb59234-97ed-4cea-849e-bee0485bdd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:13.192690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5bb59234-97ed-4cea-849e-bee0485bdd23 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:13.194046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5bb59234-97ed-4cea-849e-bee0485bdd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:13.270013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1010/2008] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:13.280672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:13.282370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:13.311215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:13.344903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967e39e-851d-44de-a688-ce2d80ab82cd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:13.349112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d169c-5452-4116-9965-fac7f2b76578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:13.349829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d169c-5452-4116-9965-fac7f2b76578 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:13.350140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d169c-5452-4116-9965-fac7f2b76578 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:13.350451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d169c-5452-4116-9965-fac7f2b76578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:13.360136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d169c-5452-4116-9965-fac7f2b76578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:13.361593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d169c-5452-4116-9965-fac7f2b76578 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:13.371426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 999/2009] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:13.669641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb59234-97ed-4cea-849e-bee0485bdd23 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:13.671849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5bb59234-97ed-4cea-849e-bee0485bdd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:13.689840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb59234-97ed-4cea-849e-bee0485bdd23 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:13.698840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:13.700774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:13.701993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:13.702367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] PATH_INFO: `/v3/projects/36cc35de74da4061b0388fff19af3af2` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:13.703957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36cc35de74da4061b0388fff19af3af2)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:13.713545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:13.778145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36cc35de74da4061b0388fff19af3af2', 'request_id': 'req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.785538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36cc35de74da4061b0388fff19af3af2', 'request_id': 'req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.795902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36cc35de74da4061b0388fff19af3af2', 'request_id': 'req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.813700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] The token cache is being invalidate because project 36cc35de74da4061b0388fff19af3af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:13.814103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.814940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.820702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1011/2010] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:13 2022] DELETE /identity/v3/projects/36cc35de74da4061b0388fff19af3af2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:13.821815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d169c-5452-4116-9965-fac7f2b76578 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:13.834025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d169c-5452-4116-9965-fac7f2b76578 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:13.909997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:13.910991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:13.911380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:13.911856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] PATH_INFO: `/v3/users/6b922ba2351d46b5a7301f198952cf6e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:13.913399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b922ba2351d46b5a7301f198952cf6e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:13.915401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:13.994433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b922ba2351d46b5a7301f198952cf6e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:13.995245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b922ba2351d46b5a7301f198952cf6e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:14.006921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b922ba2351d46b5a7301f198952cf6e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:14.008061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b922ba2351d46b5a7301f198952cf6e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:14.023588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-13b97d95-51aa-4e1a-9d2f-5d9ce3edaf8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b922ba2351d46b5a7301f198952cf6e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:14.037925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1000/2011] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:13 2022] DELETE /identity/v3/users/6b922ba2351d46b5a7301f198952cf6e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:14.863354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:14.865770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:14.876511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:14.880895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f2556-2587-43aa-af24-5cd0efdc0dd4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:14.953111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:14.955058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:14.955701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:14.956185 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] PATH_INFO: `/v3/users/46fb9dcfee35481099068771381f5227` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:14.957906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46fb9dcfee35481099068771381f5227)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:14.960539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:15.029695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46fb9dcfee35481099068771381f5227'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.031757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46fb9dcfee35481099068771381f5227'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.043377 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46fb9dcfee35481099068771381f5227'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.065913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46fb9dcfee35481099068771381f5227'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.067143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46fb9dcfee35481099068771381f5227'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.083970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1012/2012] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:14 2022] DELETE /identity/v3/users/46fb9dcfee35481099068771381f5227 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:15.089589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7589a6-ed4e-4a71-8cf0-e273d0a06735 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:15.090091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7589a6-ed4e-4a71-8cf0-e273d0a06735 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:15.090486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7589a6-ed4e-4a71-8cf0-e273d0a06735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:15.119893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-de7589a6-ed4e-4a71-8cf0-e273d0a06735 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:15.121282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-de7589a6-ed4e-4a71-8cf0-e273d0a06735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:15.220652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:15.275957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1001/2013] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:15.288765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:15.290886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:15.299946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7589a6-ed4e-4a71-8cf0-e273d0a06735 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:15.302215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-de7589a6-ed4e-4a71-8cf0-e273d0a06735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:15.321795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7589a6-ed4e-4a71-8cf0-e273d0a06735 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:15.334332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:15.335013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:15.335204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:15.335511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] PATH_INFO: `/v3/projects/55ca3bd2637e4fb5b74f7c0da43e3e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:15.337595 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=55ca3bd2637e4fb5b74f7c0da43e3e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:15.344255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:15.386110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32553d-c390-418f-9f51-d83569c8db75 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:15.412026 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d15eac-c092-47a7-9571-e3f8b62af372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:15.413038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d15eac-c092-47a7-9571-e3f8b62af372 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:15.413965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d15eac-c092-47a7-9571-e3f8b62af372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:15.414400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d15eac-c092-47a7-9571-e3f8b62af372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:15.444530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d15eac-c092-47a7-9571-e3f8b62af372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:15.445946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d15eac-c092-47a7-9571-e3f8b62af372 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:15.491115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1013/2014] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:15.513696 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55ca3bd2637e4fb5b74f7c0da43e3e7a', 'request_id': 'req-07800c1f-6280-4469-9c9f-ee7516f36d4d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.531816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55ca3bd2637e4fb5b74f7c0da43e3e7a', 'request_id': 'req-07800c1f-6280-4469-9c9f-ee7516f36d4d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.546194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55ca3bd2637e4fb5b74f7c0da43e3e7a', 'request_id': 'req-07800c1f-6280-4469-9c9f-ee7516f36d4d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.559540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] The token cache is being invalidate because project 55ca3bd2637e4fb5b74f7c0da43e3e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:15.560145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.560875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-07800c1f-6280-4469-9c9f-ee7516f36d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:15.562598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1002/2015] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:15 2022] DELETE /identity/v3/projects/55ca3bd2637e4fb5b74f7c0da43e3e7a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:15.568345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb7648-0434-4d62-9faf-917b58aab737 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:15.568679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb7648-0434-4d62-9faf-917b58aab737 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:15.568987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb7648-0434-4d62-9faf-917b58aab737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:15.590252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f1bb7648-0434-4d62-9faf-917b58aab737 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:15.591923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f1bb7648-0434-4d62-9faf-917b58aab737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:15.620332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:15.666953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1014/2016] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:15.675794 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758685aa-a7da-4bf5-924c-c3ab83aec179 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:15.676117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758685aa-a7da-4bf5-924c-c3ab83aec179 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:15.676403 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758685aa-a7da-4bf5-924c-c3ab83aec179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:15.701426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-758685aa-a7da-4bf5-924c-c3ab83aec179 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:15.702760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-758685aa-a7da-4bf5-924c-c3ab83aec179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:15.726484 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1003/2017] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:15.731229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb7648-0434-4d62-9faf-917b58aab737 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:15.750525 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb7648-0434-4d62-9faf-917b58aab737 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:15.761913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc707ae4-921e-4fc0-a02f-28893d1a1408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:15.764427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc707ae4-921e-4fc0-a02f-28893d1a1408 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:15.764813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc707ae4-921e-4fc0-a02f-28893d1a1408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:15.766298 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc707ae4-921e-4fc0-a02f-28893d1a1408 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:15.767169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc707ae4-921e-4fc0-a02f-28893d1a1408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:15.770422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc707ae4-921e-4fc0-a02f-28893d1a1408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:15.772267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc707ae4-921e-4fc0-a02f-28893d1a1408 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:15.797174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1015/2018] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:15.801287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758685aa-a7da-4bf5-924c-c3ab83aec179 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:15.811858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758685aa-a7da-4bf5-924c-c3ab83aec179 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:15.816788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba664f-65ad-400a-813f-79a7d6bdfaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:15.817559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba664f-65ad-400a-813f-79a7d6bdfaa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:15.817728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba664f-65ad-400a-813f-79a7d6bdfaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:15.818097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba664f-65ad-400a-813f-79a7d6bdfaa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:15.818834 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba664f-65ad-400a-813f-79a7d6bdfaa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:15.821077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba664f-65ad-400a-813f-79a7d6bdfaa0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:15.858885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1004/2019] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:15 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:15.863320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc707ae4-921e-4fc0-a02f-28893d1a1408 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:15.876675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc707ae4-921e-4fc0-a02f-28893d1a1408 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:15.989201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ef411-c590-4d83-9c4c-cb5717807c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:15.992540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ef411-c590-4d83-9c4c-cb5717807c6e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:15.993109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ef411-c590-4d83-9c4c-cb5717807c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:15.994891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ef411-c590-4d83-9c4c-cb5717807c6e admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:15.995954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ef411-c590-4d83-9c4c-cb5717807c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.001703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ef411-c590-4d83-9c4c-cb5717807c6e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.032943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1016/2020] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:15 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:16.036702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba664f-65ad-400a-813f-79a7d6bdfaa0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.064589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba664f-65ad-400a-813f-79a7d6bdfaa0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.073885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3e450-621e-4f56-980c-ad9e8d9b549b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.074797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3e450-621e-4f56-980c-ad9e8d9b549b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.075233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3e450-621e-4f56-980c-ad9e8d9b549b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.075641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3e450-621e-4f56-980c-ad9e8d9b549b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.076422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3e450-621e-4f56-980c-ad9e8d9b549b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.086714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3e450-621e-4f56-980c-ad9e8d9b549b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.105000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1005/2021] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:16.108932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ef411-c590-4d83-9c4c-cb5717807c6e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.128998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ef411-c590-4d83-9c4c-cb5717807c6e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.146474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8a7af-26c3-4284-8478-43c0ba0210d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.147495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8a7af-26c3-4284-8478-43c0ba0210d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.147981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8a7af-26c3-4284-8478-43c0ba0210d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.148316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8a7af-26c3-4284-8478-43c0ba0210d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.149064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8a7af-26c3-4284-8478-43c0ba0210d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.150815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8a7af-26c3-4284-8478-43c0ba0210d7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.168014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1017/2022] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:16.170858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3e450-621e-4f56-980c-ad9e8d9b549b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.173475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0cb3e450-621e-4f56-980c-ad9e8d9b549b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:16.178077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8a7af-26c3-4284-8478-43c0ba0210d7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.198340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8a7af-26c3-4284-8478-43c0ba0210d7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.198474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3e450-621e-4f56-980c-ad9e8d9b549b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.207620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44557cad-c436-4bf2-9277-c3035aa2a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.208553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.208974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44557cad-c436-4bf2-9277-c3035aa2a4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.209388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44557cad-c436-4bf2-9277-c3035aa2a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.209872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44557cad-c436-4bf2-9277-c3035aa2a4ac admin admin] PATH_INFO: `/v3/projects/1dde0d2953bb48718342af5b9630b178/users/c58d2efe81f84c498edd6459bbd69be4/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.210520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.218660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.219021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] PATH_INFO: `/v3/projects/c5ba744f426b45c1ac17bc635db87532` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.220406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5ba744f426b45c1ac17bc635db87532)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.222474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.227590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44557cad-c436-4bf2-9277-c3035aa2a4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dde0d2953bb48718342af5b9630b178, user_id=c58d2efe81f84c498edd6459bbd69be4, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.230256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44557cad-c436-4bf2-9277-c3035aa2a4ac admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.244702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1018/2023] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:16 2022] PUT /identity/v3/projects/1dde0d2953bb48718342af5b9630b178/users/c58d2efe81f84c498edd6459bbd69be4/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:16.249759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f7b6-f143-4b2f-b835-778921a15adc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.250096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f7b6-f143-4b2f-b835-778921a15adc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.250388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f7b6-f143-4b2f-b835-778921a15adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.268462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e6e4f7b6-f143-4b2f-b835-778921a15adc None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:16.269672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e6e4f7b6-f143-4b2f-b835-778921a15adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:16.278582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5ba744f426b45c1ac17bc635db87532', 'request_id': 'req-6824a490-83b5-42a2-8f09-7f79073a593a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:16.286339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5ba744f426b45c1ac17bc635db87532', 'request_id': 'req-6824a490-83b5-42a2-8f09-7f79073a593a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:16.296201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5ba744f426b45c1ac17bc635db87532', 'request_id': 'req-6824a490-83b5-42a2-8f09-7f79073a593a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:16.308806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] The token cache is being invalidate because project c5ba744f426b45c1ac17bc635db87532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:16.309100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:16.309790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6824a490-83b5-42a2-8f09-7f79073a593a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:16.311639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1006/2024] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:16 2022] DELETE /identity/v3/projects/c5ba744f426b45c1ac17bc635db87532 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:16.316643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c4103-bb29-46f2-849c-107881ad5382 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.316933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c4103-bb29-46f2-849c-107881ad5382 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.317270 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c4103-bb29-46f2-849c-107881ad5382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.347368 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2c2c4103-bb29-46f2-849c-107881ad5382 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:16.348921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2c2c4103-bb29-46f2-849c-107881ad5382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:16.353778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1019/2025] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:16.368024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1007/2026] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:16.368197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4f7b6-f143-4b2f-b835-778921a15adc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.368197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e6e4f7b6-f143-4b2f-b835-778921a15adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:16.369709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25566-6b43-4c0c-900d-e537af5d64e5 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.370030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25566-6b43-4c0c-900d-e537af5d64e5 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.370367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25566-6b43-4c0c-900d-e537af5d64e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.378753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:16.382564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:16.397236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-57b25566-6b43-4c0c-900d-e537af5d64e5 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:16.399218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-57b25566-6b43-4c0c-900d-e537af5d64e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:16.408851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1008/2027] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:16.412878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25566-6b43-4c0c-900d-e537af5d64e5 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.424720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25566-6b43-4c0c-900d-e537af5d64e5 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.430189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947155f-373d-43b5-92c2-3132af4e1a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.431310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947155f-373d-43b5-92c2-3132af4e1a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.431658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947155f-373d-43b5-92c2-3132af4e1a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.432364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947155f-373d-43b5-92c2-3132af4e1a2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.433094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947155f-373d-43b5-92c2-3132af4e1a2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:16.433416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947155f-373d-43b5-92c2-3132af4e1a2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.436250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947155f-373d-43b5-92c2-3132af4e1a2a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.440306 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4f7b6-f143-4b2f-b835-778921a15adc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.443962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d42bf-0108-4159-9b2c-6c15714df04e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.444654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d42bf-0108-4159-9b2c-6c15714df04e service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.444943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d42bf-0108-4159-9b2c-6c15714df04e service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.445355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d42bf-0108-4159-9b2c-6c15714df04e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.452546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1009/2028] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:35:16.457721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947155f-373d-43b5-92c2-3132af4e1a2a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.458262 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d42bf-0108-4159-9b2c-6c15714df04e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.460000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d42bf-0108-4159-9b2c-6c15714df04e service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.470784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947155f-373d-43b5-92c2-3132af4e1a2a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.472712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1020/2029] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:16.477131 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01e73f-9a8a-40e8-88e8-9f79ddce732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.477922 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01e73f-9a8a-40e8-88e8-9f79ddce732e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.478308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01e73f-9a8a-40e8-88e8-9f79ddce732e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.478759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01e73f-9a8a-40e8-88e8-9f79ddce732e admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.480054 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01e73f-9a8a-40e8-88e8-9f79ddce732e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.482178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01e73f-9a8a-40e8-88e8-9f79ddce732e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.523243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1010/2030] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:16 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:16.527710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d42bf-0108-4159-9b2c-6c15714df04e service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.539386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d42bf-0108-4159-9b2c-6c15714df04e service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.614911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbac04-dd40-4b7b-b72a-2b14cd09609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.615708 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbac04-dd40-4b7b-b72a-2b14cd09609d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.615990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbac04-dd40-4b7b-b72a-2b14cd09609d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.616502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbac04-dd40-4b7b-b72a-2b14cd09609d admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.617439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbac04-dd40-4b7b-b72a-2b14cd09609d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.619922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbac04-dd40-4b7b-b72a-2b14cd09609d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.659520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1021/2031] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:16 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:16.661120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01e73f-9a8a-40e8-88e8-9f79ddce732e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.674160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01e73f-9a8a-40e8-88e8-9f79ddce732e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.681824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0557f8d-1ca4-4b9b-aab4-7d15d61be22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.683133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0557f8d-1ca4-4b9b-aab4-7d15d61be22a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.683133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0557f8d-1ca4-4b9b-aab4-7d15d61be22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.683133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0557f8d-1ca4-4b9b-aab4-7d15d61be22a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.683878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0557f8d-1ca4-4b9b-aab4-7d15d61be22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.685559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0557f8d-1ca4-4b9b-aab4-7d15d61be22a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.699346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1011/2032] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:16.702282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbac04-dd40-4b7b-b72a-2b14cd09609d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.719556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbac04-dd40-4b7b-b72a-2b14cd09609d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.725319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c671677-fd30-409e-ae76-776ba80df555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.725985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c671677-fd30-409e-ae76-776ba80df555 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.726281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c671677-fd30-409e-ae76-776ba80df555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.726540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c671677-fd30-409e-ae76-776ba80df555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.727758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c671677-fd30-409e-ae76-776ba80df555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.728983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c671677-fd30-409e-ae76-776ba80df555 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.741250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1022/2033] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:16.745635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0557f8d-1ca4-4b9b-aab4-7d15d61be22a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.756448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0557f8d-1ca4-4b9b-aab4-7d15d61be22a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:16.762469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3857-2834-45bb-b982-f1bba57ae530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:16.763474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3857-2834-45bb-b982-f1bba57ae530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.763765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3857-2834-45bb-b982-f1bba57ae530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.764043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3857-2834-45bb-b982-f1bba57ae530 admin admin] PATH_INFO: `/v3/projects/6c04290d87154e48a3bf77a828b563ba/users/4f509b8a8e3c4bc3a5143d44c078d939/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.784264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f3857-2834-45bb-b982-f1bba57ae530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c04290d87154e48a3bf77a828b563ba, user_id=4f509b8a8e3c4bc3a5143d44c078d939, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:16.789062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f3857-2834-45bb-b982-f1bba57ae530 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:16.807425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1012/2034] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:16 2022] PUT /identity/v3/projects/6c04290d87154e48a3bf77a828b563ba/users/4f509b8a8e3c4bc3a5143d44c078d939/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:16.814117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8d18-8c26-4be0-b836-29ba7f5cf369 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:16.814117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8d18-8c26-4be0-b836-29ba7f5cf369 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:16.814117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d8d18-8c26-4be0-b836-29ba7f5cf369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:16.835454 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b21d8d18-8c26-4be0-b836-29ba7f5cf369 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:16.836836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b21d8d18-8c26-4be0-b836-29ba7f5cf369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:16.937711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1023/2035] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:16.988740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3857-2834-45bb-b982-f1bba57ae530 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:16.990682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9c2f3857-2834-45bb-b982-f1bba57ae530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:17.123676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3857-2834-45bb-b982-f1bba57ae530 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:17.136320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8cd53-76d6-45d6-83f4-7d4a26f678e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:17.137222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8cd53-76d6-45d6-83f4-7d4a26f678e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:17.137558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8cd53-76d6-45d6-83f4-7d4a26f678e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:17.137941 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8cd53-76d6-45d6-83f4-7d4a26f678e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:17.150575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8cd53-76d6-45d6-83f4-7d4a26f678e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:17.153020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8cd53-76d6-45d6-83f4-7d4a26f678e3 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:17.169478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1013/2036] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:17.500778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda097a-f560-4f74-8cee-8f05f8915ac6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:17.501403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda097a-f560-4f74-8cee-8f05f8915ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:17.501740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda097a-f560-4f74-8cee-8f05f8915ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:17.563000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4fda097a-f560-4f74-8cee-8f05f8915ac6 None None] MFA Rules not processed for user `c58d2efe81f84c498edd6459bbd69be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:17.564246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4fda097a-f560-4f74-8cee-8f05f8915ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:17.587249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:17.646297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1024/2037] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:17.649543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8cd53-76d6-45d6-83f4-7d4a26f678e3 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:17.652443 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-11b8cd53-76d6-45d6-83f4-7d4a26f678e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:17.684592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8cd53-76d6-45d6-83f4-7d4a26f678e3 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:17.697699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01485461-1676-4a46-a520-6860d92176f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:17.698398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01485461-1676-4a46-a520-6860d92176f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:17.700293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01485461-1676-4a46-a520-6860d92176f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:17.701370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01485461-1676-4a46-a520-6860d92176f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:17.705244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01485461-1676-4a46-a520-6860d92176f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:17.714174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01485461-1676-4a46-a520-6860d92176f4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:17.763378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1014/2038] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:17 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:17.770902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda097a-f560-4f74-8cee-8f05f8915ac6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:17.808806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda097a-f560-4f74-8cee-8f05f8915ac6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:17.916482 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d789ca8-8e6f-43b5-81e7-84bc53d14cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:17.917464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d789ca8-8e6f-43b5-81e7-84bc53d14cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:17.918253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d789ca8-8e6f-43b5-81e7-84bc53d14cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:17.919563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d789ca8-8e6f-43b5-81e7-84bc53d14cfa admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:17.921274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d789ca8-8e6f-43b5-81e7-84bc53d14cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:17.924651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d789ca8-8e6f-43b5-81e7-84bc53d14cfa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:17.962832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1025/2039] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:17 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:17.963684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01485461-1676-4a46-a520-6860d92176f4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:17.983104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01485461-1676-4a46-a520-6860d92176f4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:18.003389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def86a6-fe0e-4f3d-b309-b65092457e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:18.004377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def86a6-fe0e-4f3d-b309-b65092457e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:18.004667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def86a6-fe0e-4f3d-b309-b65092457e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:18.005489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def86a6-fe0e-4f3d-b309-b65092457e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:18.006365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def86a6-fe0e-4f3d-b309-b65092457e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:18.012936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def86a6-fe0e-4f3d-b309-b65092457e18 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:18.028995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1015/2040] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:18.033629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d789ca8-8e6f-43b5-81e7-84bc53d14cfa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:18.050418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d789ca8-8e6f-43b5-81e7-84bc53d14cfa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:18.053855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78810cfe-ff2f-4ab9-9b66-20ae1b1596aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:18.055510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78810cfe-ff2f-4ab9-9b66-20ae1b1596aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:18.057084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78810cfe-ff2f-4ab9-9b66-20ae1b1596aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:18.059329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78810cfe-ff2f-4ab9-9b66-20ae1b1596aa admin admin] PATH_INFO: `/v3/projects/df3cbd2bd8e94817ba1584161b33d101/users/0bae46342cef432bbb672b168bbf8edb/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:18.113449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78810cfe-ff2f-4ab9-9b66-20ae1b1596aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3cbd2bd8e94817ba1584161b33d101, user_id=0bae46342cef432bbb672b168bbf8edb, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:18.123347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78810cfe-ff2f-4ab9-9b66-20ae1b1596aa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:18.154494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1026/2041] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:18 2022] PUT /identity/v3/projects/df3cbd2bd8e94817ba1584161b33d101/users/0bae46342cef432bbb672b168bbf8edb/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:18.160827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def86a6-fe0e-4f3d-b309-b65092457e18 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:18.187508 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78810cfe-ff2f-4ab9-9b66-20ae1b1596aa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:18.195035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-78810cfe-ff2f-4ab9-9b66-20ae1b1596aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:18.195705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def86a6-fe0e-4f3d-b309-b65092457e18 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:18.349488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:18.359240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981a6c2-bfc1-454b-8344-5efa3329d9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:18.360285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981a6c2-bfc1-454b-8344-5efa3329d9fe admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:18.360732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981a6c2-bfc1-454b-8344-5efa3329d9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:18.361290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981a6c2-bfc1-454b-8344-5efa3329d9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:18.362958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981a6c2-bfc1-454b-8344-5efa3329d9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:18.364907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981a6c2-bfc1-454b-8344-5efa3329d9fe admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:18.377508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1016/2042] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:18.381785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981a6c2-bfc1-454b-8344-5efa3329d9fe admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:18.403337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981a6c2-bfc1-454b-8344-5efa3329d9fe admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:18.417921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd18144-2a3b-4d87-85e7-04e3597f2aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:18.418832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd18144-2a3b-4d87-85e7-04e3597f2aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:18.419116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd18144-2a3b-4d87-85e7-04e3597f2aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:18.419420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd18144-2a3b-4d87-85e7-04e3597f2aac admin admin] PATH_INFO: `/v3/projects/df3cbd2bd8e94817ba1584161b33d101/users/0bae46342cef432bbb672b168bbf8edb/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:18.425834 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd18144-2a3b-4d87-85e7-04e3597f2aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3cbd2bd8e94817ba1584161b33d101, user_id=0bae46342cef432bbb672b168bbf8edb, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:18.429901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd18144-2a3b-4d87-85e7-04e3597f2aac admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:18.433598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78810cfe-ff2f-4ab9-9b66-20ae1b1596aa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:18.443450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1017/2043] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:18 2022] PUT /identity/v3/projects/df3cbd2bd8e94817ba1584161b33d101/users/0bae46342cef432bbb672b168bbf8edb/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:18.496181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:18.524726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:18.525770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:18.526086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:18.526365 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] PATH_INFO: `/v3/users/ba34270bf2fa4588937b7f9a22ee9c7d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:18.530597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba34270bf2fa4588937b7f9a22ee9c7d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:18.534019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:18.628335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba34270bf2fa4588937b7f9a22ee9c7d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:18.629982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba34270bf2fa4588937b7f9a22ee9c7d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:18.642427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba34270bf2fa4588937b7f9a22ee9c7d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:18.668558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba34270bf2fa4588937b7f9a22ee9c7d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:18.669675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba34270bf2fa4588937b7f9a22ee9c7d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:18.684751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1027/2044] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:18 2022] DELETE /identity/v3/users/ba34270bf2fa4588937b7f9a22ee9c7d => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:18.689542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30987c-afd4-4d8c-aa1e-e35699f925b9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:18.690064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30987c-afd4-4d8c-aa1e-e35699f925b9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:18.690542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30987c-afd4-4d8c-aa1e-e35699f925b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:18.723207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0c30987c-afd4-4d8c-aa1e-e35699f925b9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:18.725684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0c30987c-afd4-4d8c-aa1e-e35699f925b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:18.855100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1018/2045] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:18.906469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:18.907958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:19.040708 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6941d7-3a96-4c63-a113-77057d079949 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:19.054062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef9702-c009-47ce-9b87-58feb7be1c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:19.056215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef9702-c009-47ce-9b87-58feb7be1c31 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:19.056659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef9702-c009-47ce-9b87-58feb7be1c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:19.057029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef9702-c009-47ce-9b87-58feb7be1c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:19.074674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef9702-c009-47ce-9b87-58feb7be1c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:19.076238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef9702-c009-47ce-9b87-58feb7be1c31 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:19.089486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1028/2046] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:20.286156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30987c-afd4-4d8c-aa1e-e35699f925b9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:20.288117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0c30987c-afd4-4d8c-aa1e-e35699f925b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:20.313511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30987c-afd4-4d8c-aa1e-e35699f925b9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:20.336943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:20.337830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:20.338129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:20.338391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] PATH_INFO: `/v3/projects/4125ba5fb9944574a1d31f4f45c4dbd0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:20.340212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4125ba5fb9944574a1d31f4f45c4dbd0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:20.342166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:20.484664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4125ba5fb9944574a1d31f4f45c4dbd0', 'request_id': 'req-e39c3de4-4225-487e-ab62-46018b7f41a8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.505307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4125ba5fb9944574a1d31f4f45c4dbd0', 'request_id': 'req-e39c3de4-4225-487e-ab62-46018b7f41a8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.509892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0caa2c-6d40-4b34-8d8c-0d4543135f68 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:20.512039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0caa2c-6d40-4b34-8d8c-0d4543135f68 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:20.512519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0caa2c-6d40-4b34-8d8c-0d4543135f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:20.520728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4125ba5fb9944574a1d31f4f45c4dbd0', 'request_id': 'req-e39c3de4-4225-487e-ab62-46018b7f41a8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.539886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] The token cache is being invalidate because project 4125ba5fb9944574a1d31f4f45c4dbd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:20.540454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.541954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.544390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1019/2047] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:20 2022] DELETE /identity/v3/projects/4125ba5fb9944574a1d31f4f45c4dbd0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:20.550936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:20.552743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:20.597388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7a0caa2c-6d40-4b34-8d8c-0d4543135f68 None None] MFA Rules not processed for user `0bae46342cef432bbb672b168bbf8edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:20.598635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7a0caa2c-6d40-4b34-8d8c-0d4543135f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:20.698706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:20.765293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c3de4-4225-487e-ab62-46018b7f41a8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:20.773246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1029/2048] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:20.773984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:20.775172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:20.776835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:20.777213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] PATH_INFO: `/v3/users/4de91929e4b64a19bb4cee7ecb39752f` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:20.778645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de91929e4b64a19bb4cee7ecb39752f)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:20.780372 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0caa2c-6d40-4b34-8d8c-0d4543135f68 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:20.790882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:20.791622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:20.792246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7a0caa2c-6d40-4b34-8d8c-0d4543135f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:20.857240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de91929e4b64a19bb4cee7ecb39752f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.858263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de91929e4b64a19bb4cee7ecb39752f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.871250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de91929e4b64a19bb4cee7ecb39752f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.872193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de91929e4b64a19bb4cee7ecb39752f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.873065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0caa2c-6d40-4b34-8d8c-0d4543135f68 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:20.878013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c32f64-5af7-4143-bba7-6f7e3e625ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:20.879126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c32f64-5af7-4143-bba7-6f7e3e625ced service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:20.879448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c32f64-5af7-4143-bba7-6f7e3e625ced service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:20.880144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c32f64-5af7-4143-bba7-6f7e3e625ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:20.882211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-99c32f64-5af7-4143-bba7-6f7e3e625ced service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:20.889106 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de91929e4b64a19bb4cee7ecb39752f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:20.913104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1020/2049] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:20 2022] DELETE /identity/v3/users/4de91929e4b64a19bb4cee7ecb39752f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:20.952363 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c32f64-5af7-4143-bba7-6f7e3e625ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:20.953879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c32f64-5af7-4143-bba7-6f7e3e625ced service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:20.977621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1030/2050] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:35:21.477077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:21.478792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:21.593646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c21fad-7f79-4984-b467-b4511f3b8f6d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:21.601327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:21.602229 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:21.605581 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:21.605942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] PATH_INFO: `/v3/projects/b62abe52aef048f597242629585ed5b2` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:21.607441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62abe52aef048f597242629585ed5b2)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:21.609463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:21.683923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62abe52aef048f597242629585ed5b2', 'request_id': 'req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:21.708133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62abe52aef048f597242629585ed5b2', 'request_id': 'req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:21.729669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62abe52aef048f597242629585ed5b2', 'request_id': 'req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:21.772246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] The token cache is being invalidate because project b62abe52aef048f597242629585ed5b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:21.772548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:21.773998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0e1f2f22-841f-4ed4-a6d8-2e7be44bd6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:21.782782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1021/2051] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:21 2022] DELETE /identity/v3/projects/b62abe52aef048f597242629585ed5b2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:21.787817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cd48b-8222-4055-9cff-47faac880d6e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:21.788730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cd48b-8222-4055-9cff-47faac880d6e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:21.790153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cd48b-8222-4055-9cff-47faac880d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:21.858628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-041cd48b-8222-4055-9cff-47faac880d6e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:21.862664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-041cd48b-8222-4055-9cff-47faac880d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:22.076986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1031/2052] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:22.080314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a3bf-069e-4d0b-a87b-b901949484e4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:22.080871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a3bf-069e-4d0b-a87b-b901949484e4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:22.081371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0a3bf-069e-4d0b-a87b-b901949484e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:22.122687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b3b0a3bf-069e-4d0b-a87b-b901949484e4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:22.125399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b3b0a3bf-069e-4d0b-a87b-b901949484e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:22.152568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1022/2053] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:22.154200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cd48b-8222-4055-9cff-47faac880d6e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:22.167754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cd48b-8222-4055-9cff-47faac880d6e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:22.182355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a2812-1144-41bc-8d10-e5bca02a9c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:22.183220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a2812-1144-41bc-8d10-e5bca02a9c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:22.183553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a2812-1144-41bc-8d10-e5bca02a9c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:22.183840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a2812-1144-41bc-8d10-e5bca02a9c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:22.184593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a2812-1144-41bc-8d10-e5bca02a9c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:22.184907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a2812-1144-41bc-8d10-e5bca02a9c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:22.186686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a2812-1144-41bc-8d10-e5bca02a9c64 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:22.220567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1032/2054] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:22.225770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0a3bf-069e-4d0b-a87b-b901949484e4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:22.252289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0a3bf-069e-4d0b-a87b-b901949484e4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:22.258656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb84129-cff1-4546-b1fa-16a185f70255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:22.259494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb84129-cff1-4546-b1fa-16a185f70255 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:22.259837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb84129-cff1-4546-b1fa-16a185f70255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:22.261298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb84129-cff1-4546-b1fa-16a185f70255 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:22.262621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb84129-cff1-4546-b1fa-16a185f70255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:22.266372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb84129-cff1-4546-b1fa-16a185f70255 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:22.273413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:22.358861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1023/2055] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:22 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:22.363527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a2812-1144-41bc-8d10-e5bca02a9c64 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:22.389665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a2812-1144-41bc-8d10-e5bca02a9c64 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:22.459453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb84129-cff1-4546-b1fa-16a185f70255 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:22.461791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0fb84129-cff1-4546-b1fa-16a185f70255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:22.485850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:22.584508 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733cc72-9488-4ff1-8509-5d722a3646ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:22.585644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733cc72-9488-4ff1-8509-5d722a3646ad admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:22.596549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733cc72-9488-4ff1-8509-5d722a3646ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:22.596549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733cc72-9488-4ff1-8509-5d722a3646ad admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:22.597704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733cc72-9488-4ff1-8509-5d722a3646ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:22.603191 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733cc72-9488-4ff1-8509-5d722a3646ad admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:22.665842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1033/2056] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:22 2022] POST /identity/v3/users => generated 282 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:22.667171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733cc72-9488-4ff1-8509-5d722a3646ad admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:22.672699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb84129-cff1-4546-b1fa-16a185f70255 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:22.681263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870da0a-8e0e-45ef-a62e-2c86091eb3bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:22.681900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870da0a-8e0e-45ef-a62e-2c86091eb3bf service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:22.682185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870da0a-8e0e-45ef-a62e-2c86091eb3bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:22.682460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870da0a-8e0e-45ef-a62e-2c86091eb3bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:22.686583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0870da0a-8e0e-45ef-a62e-2c86091eb3bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:22.699500 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733cc72-9488-4ff1-8509-5d722a3646ad admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:22.715415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2e213-290f-4451-948a-a768ab0e9d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:22.716334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2e213-290f-4451-948a-a768ab0e9d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:22.716724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2e213-290f-4451-948a-a768ab0e9d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:22.717128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2e213-290f-4451-948a-a768ab0e9d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:22.719344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2e213-290f-4451-948a-a768ab0e9d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:22.721383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2e213-290f-4451-948a-a768ab0e9d17 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:22.751677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1034/2057] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:22.752791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2e213-290f-4451-948a-a768ab0e9d17 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:22.776920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2e213-290f-4451-948a-a768ab0e9d17 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:22.793546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4a0e1-863c-4f62-812f-8506c7606378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:22.794362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4a0e1-863c-4f62-812f-8506c7606378 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:22.795072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4a0e1-863c-4f62-812f-8506c7606378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:22.796852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4a0e1-863c-4f62-812f-8506c7606378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:22.797871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4a0e1-863c-4f62-812f-8506c7606378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:22.799989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4a0e1-863c-4f62-812f-8506c7606378 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:22.839536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1035/2058] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:22.842047 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4a0e1-863c-4f62-812f-8506c7606378 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:22.851142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870da0a-8e0e-45ef-a62e-2c86091eb3bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:22.854426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870da0a-8e0e-45ef-a62e-2c86091eb3bf service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:22.867942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4a0e1-863c-4f62-812f-8506c7606378 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:22.881378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1024/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:22.884675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d832823-0214-46c5-9b12-163ef81c81c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:22.886054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d832823-0214-46c5-9b12-163ef81c81c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:22.886609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d832823-0214-46c5-9b12-163ef81c81c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:22.887076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d832823-0214-46c5-9b12-163ef81c81c3 admin admin] PATH_INFO: `/v3/projects/24991d3d422c4effaeb62e972873d8eb/users/749eb74dd2da416d85eff33793fff74a/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:22.939323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d832823-0214-46c5-9b12-163ef81c81c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24991d3d422c4effaeb62e972873d8eb, user_id=749eb74dd2da416d85eff33793fff74a, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:22.943034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d832823-0214-46c5-9b12-163ef81c81c3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:22.983345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1036/2060] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:22 2022] PUT /identity/v3/projects/24991d3d422c4effaeb62e972873d8eb/users/749eb74dd2da416d85eff33793fff74a/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:22.984537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c4c86-4c37-4f74-947a-5495beec1534 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:22.984836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c4c86-4c37-4f74-947a-5495beec1534 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:22.985170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c4c86-4c37-4f74-947a-5495beec1534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:23.050480 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5d6c4c86-4c37-4f74-947a-5495beec1534 None None] MFA Rules not processed for user `749eb74dd2da416d85eff33793fff74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:23.053766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d6c4c86-4c37-4f74-947a-5495beec1534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:23.195353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1025/2061] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:22 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:23.198834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d832823-0214-46c5-9b12-163ef81c81c3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:23.426901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d832823-0214-46c5-9b12-163ef81c81c3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:23.440753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6911bd-efcf-4516-a468-4d7d9fc1a0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:23.445163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6911bd-efcf-4516-a468-4d7d9fc1a0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:23.446301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6911bd-efcf-4516-a468-4d7d9fc1a0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:23.447916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6911bd-efcf-4516-a468-4d7d9fc1a0a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:23.448899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6911bd-efcf-4516-a468-4d7d9fc1a0a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:23.457829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6911bd-efcf-4516-a468-4d7d9fc1a0a3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:23.561713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1037/2062] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:23 2022] POST /identity/v3/projects => generated 363 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:23.563920 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c4c86-4c37-4f74-947a-5495beec1534 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:23.576291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6911bd-efcf-4516-a468-4d7d9fc1a0a3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:23.591972 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ce6911bd-efcf-4516-a468-4d7d9fc1a0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:23.597354 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c4c86-4c37-4f74-947a-5495beec1534 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:23.657695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:23.691545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:23.712367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6911bd-efcf-4516-a468-4d7d9fc1a0a3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:23.720063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:23.721001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:23.723774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:23.724133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] PATH_INFO: `/v3/users/e945141497a9424e910b0d4c77dd32e7` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:23.728868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e945141497a9424e910b0d4c77dd32e7)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:23.737529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:23.760185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8329-3c5d-47ce-af71-2ead934da672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:23.760889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8329-3c5d-47ce-af71-2ead934da672 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:23.761177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8329-3c5d-47ce-af71-2ead934da672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:23.761458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd8329-3c5d-47ce-af71-2ead934da672 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:23.762208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd8329-3c5d-47ce-af71-2ead934da672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:23.773724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd8329-3c5d-47ce-af71-2ead934da672 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:23.826375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1026/2063] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:23 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:23.837573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8329-3c5d-47ce-af71-2ead934da672 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:23.863145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd8329-3c5d-47ce-af71-2ead934da672 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:23.869620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e945141497a9424e910b0d4c77dd32e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:23.870850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e945141497a9424e910b0d4c77dd32e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:23.878398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef1578-bee4-450e-b221-f8346b4e1935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:23.879117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef1578-bee4-450e-b221-f8346b4e1935 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:23.880626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef1578-bee4-450e-b221-f8346b4e1935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:23.882306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef1578-bee4-450e-b221-f8346b4e1935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:23.883347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef1578-bee4-450e-b221-f8346b4e1935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:23.885616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef1578-bee4-450e-b221-f8346b4e1935 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:23.891094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e945141497a9424e910b0d4c77dd32e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:23.920912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e945141497a9424e910b0d4c77dd32e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:23.921954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1027/2064] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:23.923258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-727f8dd3-252e-4899-b2ad-a47f7622b260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e945141497a9424e910b0d4c77dd32e7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:23.925616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef1578-bee4-450e-b221-f8346b4e1935 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:23.929998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5cef1578-bee4-450e-b221-f8346b4e1935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:23.959990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1038/2065] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:23 2022] DELETE /identity/v3/users/e945141497a9424e910b0d4c77dd32e7 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:23.964843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad547639-449a-46ad-aae2-fa8b9e3acfef None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:23.966844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad547639-449a-46ad-aae2-fa8b9e3acfef None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:23.967461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef1578-bee4-450e-b221-f8346b4e1935 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:23.968469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad547639-449a-46ad-aae2-fa8b9e3acfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:24.007760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ad547639-449a-46ad-aae2-fa8b9e3acfef None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:24.009658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ad547639-449a-46ad-aae2-fa8b9e3acfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:24.104241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd33892-29ff-416b-9ce2-4a0a0d3d37d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:24.105283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd33892-29ff-416b-9ce2-4a0a0d3d37d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:24.106112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd33892-29ff-416b-9ce2-4a0a0d3d37d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:24.106112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd33892-29ff-416b-9ce2-4a0a0d3d37d5 admin admin] PATH_INFO: `/v3/projects/1b8bb6db70444206b7dc1764cc9960ef/users/50b167cdc4c44b6bb91b877c1ee23c1f/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:24.128804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd33892-29ff-416b-9ce2-4a0a0d3d37d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b8bb6db70444206b7dc1764cc9960ef, user_id=50b167cdc4c44b6bb91b877c1ee23c1f, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:24.132927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd33892-29ff-416b-9ce2-4a0a0d3d37d5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:24.170283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1028/2066] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:23 2022] PUT /identity/v3/projects/1b8bb6db70444206b7dc1764cc9960ef/users/50b167cdc4c44b6bb91b877c1ee23c1f/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:24.174436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd33892-29ff-416b-9ce2-4a0a0d3d37d5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:24.184651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1039/2067] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:24.254295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad547639-449a-46ad-aae2-fa8b9e3acfef None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:24.260663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ad547639-449a-46ad-aae2-fa8b9e3acfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:24.316482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd33892-29ff-416b-9ce2-4a0a0d3d37d5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:24.331098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83b28-75b2-4e49-8505-ff646f9eff57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:24.331842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83b28-75b2-4e49-8505-ff646f9eff57 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:24.332261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83b28-75b2-4e49-8505-ff646f9eff57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:24.332551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83b28-75b2-4e49-8505-ff646f9eff57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:24.333309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a83b28-75b2-4e49-8505-ff646f9eff57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:24.338156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a83b28-75b2-4e49-8505-ff646f9eff57 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:24.364365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1029/2068] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:24.368565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83b28-75b2-4e49-8505-ff646f9eff57 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:24.380095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad547639-449a-46ad-aae2-fa8b9e3acfef None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:24.380731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83b28-75b2-4e49-8505-ff646f9eff57 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:24.387502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631837d2-5a91-454b-aaae-29bb497554bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:24.388665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631837d2-5a91-454b-aaae-29bb497554bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:24.389120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631837d2-5a91-454b-aaae-29bb497554bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:24.389658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631837d2-5a91-454b-aaae-29bb497554bc admin admin] PATH_INFO: `/v3/projects/1b8bb6db70444206b7dc1764cc9960ef/users/50b167cdc4c44b6bb91b877c1ee23c1f/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:24.397017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631837d2-5a91-454b-aaae-29bb497554bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b8bb6db70444206b7dc1764cc9960ef, user_id=50b167cdc4c44b6bb91b877c1ee23c1f, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:24.400540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631837d2-5a91-454b-aaae-29bb497554bc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:24.408385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:24.424643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1030/2069] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:24 2022] PUT /identity/v3/projects/1b8bb6db70444206b7dc1764cc9960ef/users/50b167cdc4c44b6bb91b877c1ee23c1f/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:24.448800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d24708-0042-491f-a2e4-5449d637f088 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:24.450114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d24708-0042-491f-a2e4-5449d637f088 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:24.450576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d24708-0042-491f-a2e4-5449d637f088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:24.467417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a0485-dc15-401f-aa5c-babf636ecb25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:24.468192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a0485-dc15-401f-aa5c-babf636ecb25 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:24.468480 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a0485-dc15-401f-aa5c-babf636ecb25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:24.468759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a0485-dc15-401f-aa5c-babf636ecb25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:24.488232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a0485-dc15-401f-aa5c-babf636ecb25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:24.490081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160a0485-dc15-401f-aa5c-babf636ecb25 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:24.509505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:24.512012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1040/2070] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:24.555785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c5d24708-0042-491f-a2e4-5449d637f088 None None] MFA Rules not processed for user `50b167cdc4c44b6bb91b877c1ee23c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:24.558908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c5d24708-0042-491f-a2e4-5449d637f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:24.775335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1031/2071] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:24 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:24.795596 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a0485-dc15-401f-aa5c-babf636ecb25 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:24.797195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-160a0485-dc15-401f-aa5c-babf636ecb25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:24.953445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160a0485-dc15-401f-aa5c-babf636ecb25 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:25.045522 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66cf16-fa09-4747-b171-6743089d5f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:25.045522 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66cf16-fa09-4747-b171-6743089d5f6f service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:25.045522 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66cf16-fa09-4747-b171-6743089d5f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:25.045522 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66cf16-fa09-4747-b171-6743089d5f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:25.061705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66cf16-fa09-4747-b171-6743089d5f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:25.063186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66cf16-fa09-4747-b171-6743089d5f6f service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:25.082013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1041/2072] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:24 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:25.230232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d24708-0042-491f-a2e4-5449d637f088 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:25.236282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c5d24708-0042-491f-a2e4-5449d637f088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:25.242902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66cf16-fa09-4747-b171-6743089d5f6f service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:25.252907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66cf16-fa09-4747-b171-6743089d5f6f service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:25.257479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ad2ae-b0b1-4232-afb7-653838968928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:25.258066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ad2ae-b0b1-4232-afb7-653838968928 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:25.258366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ad2ae-b0b1-4232-afb7-653838968928 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:25.258675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ad2ae-b0b1-4232-afb7-653838968928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:25.260533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-350ad2ae-b0b1-4232-afb7-653838968928 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:25.368235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d24708-0042-491f-a2e4-5449d637f088 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:25.384865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:25.386216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:25.386504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:25.391179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] PATH_INFO: `/v3/projects/e59a8c977471441e8d569ab1ba8f472b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:25.392609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e59a8c977471441e8d569ab1ba8f472b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:25.395161 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:25.437418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ad2ae-b0b1-4232-afb7-653838968928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:25.438934 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ad2ae-b0b1-4232-afb7-653838968928 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:25.469879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1042/2073] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:25 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:25.499876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e59a8c977471441e8d569ab1ba8f472b', 'request_id': 'req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:25.515565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e59a8c977471441e8d569ab1ba8f472b', 'request_id': 'req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:25.532993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e59a8c977471441e8d569ab1ba8f472b', 'request_id': 'req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:25.558627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] The token cache is being invalidate because project e59a8c977471441e8d569ab1ba8f472b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:25.558907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:25.559914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:25.561817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1032/2074] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:25 2022] DELETE /identity/v3/projects/e59a8c977471441e8d569ab1ba8f472b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:25.970235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91d351-e687-4307-a9de-a249a4671668 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:25.970716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91d351-e687-4307-a9de-a249a4671668 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:25.970838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91d351-e687-4307-a9de-a249a4671668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:26.000280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0e91d351-e687-4307-a9de-a249a4671668 None None] MFA Rules not processed for user `4f509b8a8e3c4bc3a5143d44c078d939`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:26.001977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0e91d351-e687-4307-a9de-a249a4671668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:26.020489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:26.074050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1043/2075] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:26.078673 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:26.080152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:26.109491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:26.148641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6c6cc-bb9e-47e7-81aa-e671878469fb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:26.154243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115fb0d-b56c-4803-8c36-e42ae19aedd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:26.154933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115fb0d-b56c-4803-8c36-e42ae19aedd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:26.155240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115fb0d-b56c-4803-8c36-e42ae19aedd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:26.155563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115fb0d-b56c-4803-8c36-e42ae19aedd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:26.156256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115fb0d-b56c-4803-8c36-e42ae19aedd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:26.158205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115fb0d-b56c-4803-8c36-e42ae19aedd2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:26.191730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1033/2076] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:26 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:26.193636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91d351-e687-4307-a9de-a249a4671668 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:26.213336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91d351-e687-4307-a9de-a249a4671668 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:26.287774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a927b08-74f4-4bcd-a54b-3a7eae6deace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:26.288465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927b08-74f4-4bcd-a54b-3a7eae6deace admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:26.288749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927b08-74f4-4bcd-a54b-3a7eae6deace admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:26.289027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a927b08-74f4-4bcd-a54b-3a7eae6deace admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:26.289719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a927b08-74f4-4bcd-a54b-3a7eae6deace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:26.291711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a927b08-74f4-4bcd-a54b-3a7eae6deace admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:26.314923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1044/2077] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:26 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:26.319936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115fb0d-b56c-4803-8c36-e42ae19aedd2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:26.331566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115fb0d-b56c-4803-8c36-e42ae19aedd2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:26.336969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dcc04-8e51-413e-92e6-5ae0eb57c21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:26.337866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dcc04-8e51-413e-92e6-5ae0eb57c21c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:26.338211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dcc04-8e51-413e-92e6-5ae0eb57c21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:26.338557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dcc04-8e51-413e-92e6-5ae0eb57c21c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:26.339395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dcc04-8e51-413e-92e6-5ae0eb57c21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:26.341145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dcc04-8e51-413e-92e6-5ae0eb57c21c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:26.353466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1034/2078] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:26.357448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a927b08-74f4-4bcd-a54b-3a7eae6deace admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:26.368749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a927b08-74f4-4bcd-a54b-3a7eae6deace admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:26.374794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ef569-f5ec-49c1-b236-8334374f8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:26.375990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ef569-f5ec-49c1-b236-8334374f8db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:26.376626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ef569-f5ec-49c1-b236-8334374f8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:26.376858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ef569-f5ec-49c1-b236-8334374f8db2 admin admin] PATH_INFO: `/v3/projects/49622e8284454d679cacdd88b2f2344a/users/5c014213af9f44869af94ce258d265e4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:26.394924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ef569-f5ec-49c1-b236-8334374f8db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49622e8284454d679cacdd88b2f2344a, user_id=5c014213af9f44869af94ce258d265e4, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:26.399047 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ef569-f5ec-49c1-b236-8334374f8db2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:26.417068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1045/2079] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:26 2022] PUT /identity/v3/projects/49622e8284454d679cacdd88b2f2344a/users/5c014213af9f44869af94ce258d265e4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:26.421622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dcc04-8e51-413e-92e6-5ae0eb57c21c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:26.433272 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dcc04-8e51-413e-92e6-5ae0eb57c21c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:26.494323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfd7b4-fddb-420e-8bb2-02af85d5201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:26.495013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfd7b4-fddb-420e-8bb2-02af85d5201b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:26.495323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfd7b4-fddb-420e-8bb2-02af85d5201b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:26.495591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfd7b4-fddb-420e-8bb2-02af85d5201b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:26.496313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cfd7b4-fddb-420e-8bb2-02af85d5201b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:26.498149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cfd7b4-fddb-420e-8bb2-02af85d5201b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:26.509370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1035/2080] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:26.514208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ef569-f5ec-49c1-b236-8334374f8db2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:26.524436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ef569-f5ec-49c1-b236-8334374f8db2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:26.529015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71775-21b2-4050-b033-aaf4b82cd383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:26.529900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71775-21b2-4050-b033-aaf4b82cd383 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:26.530176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71775-21b2-4050-b033-aaf4b82cd383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:26.530447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71775-21b2-4050-b033-aaf4b82cd383 admin admin] PATH_INFO: `/v3/projects/49622e8284454d679cacdd88b2f2344a/users/5c014213af9f44869af94ce258d265e4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:26.532449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd71775-21b2-4050-b033-aaf4b82cd383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49622e8284454d679cacdd88b2f2344a, user_id=5c014213af9f44869af94ce258d265e4, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:26.535088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd71775-21b2-4050-b033-aaf4b82cd383 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:26.548234 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1046/2081] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:26 2022] PUT /identity/v3/projects/49622e8284454d679cacdd88b2f2344a/users/5c014213af9f44869af94ce258d265e4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:32.671506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03338819-0df1-4757-b42d-185083e48202 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:32.671977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03338819-0df1-4757-b42d-185083e48202 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:32.672208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03338819-0df1-4757-b42d-185083e48202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:32.719189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-03338819-0df1-4757-b42d-185083e48202 None None] MFA Rules not processed for user `5c014213af9f44869af94ce258d265e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:32.720604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-03338819-0df1-4757-b42d-185083e48202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:32.806500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1036/2082] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:32.820524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71775-21b2-4050-b033-aaf4b82cd383 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:32.823766 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cfd71775-21b2-4050-b033-aaf4b82cd383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:32.915095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71775-21b2-4050-b033-aaf4b82cd383 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:32.921044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24e4ef-a346-4006-9a78-558b14d116fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:32.921642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24e4ef-a346-4006-9a78-558b14d116fd service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:32.921922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24e4ef-a346-4006-9a78-558b14d116fd service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:32.922196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24e4ef-a346-4006-9a78-558b14d116fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:32.926906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:33.000438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24e4ef-a346-4006-9a78-558b14d116fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:33.002247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24e4ef-a346-4006-9a78-558b14d116fd service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:33.014082 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1047/2083] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:34.315542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6d9e-4f10-4637-b42e-cc889ddf0c11 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:34.316029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6d9e-4f10-4637-b42e-cc889ddf0c11 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:34.316259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6d9e-4f10-4637-b42e-cc889ddf0c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:34.344344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d92b6d9e-4f10-4637-b42e-cc889ddf0c11 None None] MFA Rules not processed for user `4f509b8a8e3c4bc3a5143d44c078d939`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:34.344344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d92b6d9e-4f10-4637-b42e-cc889ddf0c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:34.361221 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1037/2084] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:39.432383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24e4ef-a346-4006-9a78-558b14d116fd service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:39.434104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5b24e4ef-a346-4006-9a78-558b14d116fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:39.510460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24e4ef-a346-4006-9a78-558b14d116fd service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:39.516887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:39.517785 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:39.518059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:39.518328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] PATH_INFO: `/v3/users/749eb74dd2da416d85eff33793fff74a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:39.519561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=749eb74dd2da416d85eff33793fff74a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:39.521672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:39.575515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '749eb74dd2da416d85eff33793fff74a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:39.576158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '749eb74dd2da416d85eff33793fff74a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:39.586284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '749eb74dd2da416d85eff33793fff74a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:39.600538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '749eb74dd2da416d85eff33793fff74a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:39.601264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '749eb74dd2da416d85eff33793fff74a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:39.614143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1048/2085] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:39 2022] DELETE /identity/v3/users/749eb74dd2da416d85eff33793fff74a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:39.619081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4fc84-f918-4acf-8057-6e9b21bd1e70 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:39.619611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4fc84-f918-4acf-8057-6e9b21bd1e70 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:39.619814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4fc84-f918-4acf-8057-6e9b21bd1e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:39.638465 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f9d4fc84-f918-4acf-8057-6e9b21bd1e70 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:39.639836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f9d4fc84-f918-4acf-8057-6e9b21bd1e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:39.652401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:39.711367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1038/2086] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:39.718197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:39.719848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:39.766879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:39.807791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aa28f-5960-4ab4-aa5f-8656d49e8e32 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:39.814915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbce770-42a7-4d13-a6a4-e4d2f3c272f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:39.815781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbce770-42a7-4d13-a6a4-e4d2f3c272f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:39.815781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbce770-42a7-4d13-a6a4-e4d2f3c272f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:39.815992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbce770-42a7-4d13-a6a4-e4d2f3c272f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:39.828792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbce770-42a7-4d13-a6a4-e4d2f3c272f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:39.830391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbce770-42a7-4d13-a6a4-e4d2f3c272f8 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:39.846144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1049/2087] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:40.343820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4fc84-f918-4acf-8057-6e9b21bd1e70 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:40.346191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f9d4fc84-f918-4acf-8057-6e9b21bd1e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:40.362065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4fc84-f918-4acf-8057-6e9b21bd1e70 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:40.367880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:40.368921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:40.369050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:40.369483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] PATH_INFO: `/v3/projects/24991d3d422c4effaeb62e972873d8eb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:40.370693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=24991d3d422c4effaeb62e972873d8eb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:40.372746 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:40.428203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24991d3d422c4effaeb62e972873d8eb', 'request_id': 'req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.437648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24991d3d422c4effaeb62e972873d8eb', 'request_id': 'req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.444475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24991d3d422c4effaeb62e972873d8eb', 'request_id': 'req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.455026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] The token cache is being invalidate because project 24991d3d422c4effaeb62e972873d8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:40.455332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.455982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.460937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1039/2088] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:40 2022] DELETE /identity/v3/projects/24991d3d422c4effaeb62e972873d8eb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:40.462104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbce770-42a7-4d13-a6a4-e4d2f3c272f8 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:40.472412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbce770-42a7-4d13-a6a4-e4d2f3c272f8 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:40.536581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:40.537462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:40.537754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:40.538030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] PATH_INFO: `/v3/users/50b167cdc4c44b6bb91b877c1ee23c1f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:40.539274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b167cdc4c44b6bb91b877c1ee23c1f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:40.541267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:40.595453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b167cdc4c44b6bb91b877c1ee23c1f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.596036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b167cdc4c44b6bb91b877c1ee23c1f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.605345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b167cdc4c44b6bb91b877c1ee23c1f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.617397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b167cdc4c44b6bb91b877c1ee23c1f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.618032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b76d2f56-3dc6-41d4-b2fe-03e470fac897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b167cdc4c44b6bb91b877c1ee23c1f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:40.629954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1050/2089] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:40 2022] DELETE /identity/v3/users/50b167cdc4c44b6bb91b877c1ee23c1f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:40.912961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:40.914607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:40.930557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cfaf4-4ae3-4eec-8093-8702bc00e2ee admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:41.002029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:41.002937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.003225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.003710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] PATH_INFO: `/v3/projects/1b8bb6db70444206b7dc1764cc9960ef` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.005116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b8bb6db70444206b7dc1764cc9960ef)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:41.008014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:41.063561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:41.073533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b8bb6db70444206b7dc1764cc9960ef', 'request_id': 'req-27cc2a49-28b2-4782-891e-8d45d008679d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:41.088236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b8bb6db70444206b7dc1764cc9960ef', 'request_id': 'req-27cc2a49-28b2-4782-891e-8d45d008679d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:41.107008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b8bb6db70444206b7dc1764cc9960ef', 'request_id': 'req-27cc2a49-28b2-4782-891e-8d45d008679d'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:41.138482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] The token cache is being invalidate because project 1b8bb6db70444206b7dc1764cc9960ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:41.138482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:41.139846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-27cc2a49-28b2-4782-891e-8d45d008679d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:41.141558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1040/2090] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:40 2022] DELETE /identity/v3/projects/1b8bb6db70444206b7dc1764cc9960ef => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:41.146428 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c099a-2f9a-4e0c-87ea-5d31ba63d414 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.146757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c099a-2f9a-4e0c-87ea-5d31ba63d414 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.147099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c099a-2f9a-4e0c-87ea-5d31ba63d414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.186830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5f7c099a-2f9a-4e0c-87ea-5d31ba63d414 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:41.189173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5f7c099a-2f9a-4e0c-87ea-5d31ba63d414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:41.244226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:41.277901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1051/2091] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:41.286423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d610ea5-26b1-49e3-bdc5-5bff64dcee4a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.289763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d610ea5-26b1-49e3-bdc5-5bff64dcee4a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.290361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d610ea5-26b1-49e3-bdc5-5bff64dcee4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.309870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2d610ea5-26b1-49e3-bdc5-5bff64dcee4a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:41.311127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2d610ea5-26b1-49e3-bdc5-5bff64dcee4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:41.319262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1041/2092] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:41.323577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c099a-2f9a-4e0c-87ea-5d31ba63d414 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:41.334448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c099a-2f9a-4e0c-87ea-5d31ba63d414 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:41.339322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f8062-cf18-473b-b8a5-2aa15872a848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:41.340033 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f8062-cf18-473b-b8a5-2aa15872a848 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.340316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f8062-cf18-473b-b8a5-2aa15872a848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.340595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f8062-cf18-473b-b8a5-2aa15872a848 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.341354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f8062-cf18-473b-b8a5-2aa15872a848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:41.341662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f8062-cf18-473b-b8a5-2aa15872a848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:41.343319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f8062-cf18-473b-b8a5-2aa15872a848 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:41.356601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1052/2093] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:41.361571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d610ea5-26b1-49e3-bdc5-5bff64dcee4a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:41.371766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d610ea5-26b1-49e3-bdc5-5bff64dcee4a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:41.376860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dff6ed-4593-4eb1-8754-509f235c4fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:41.377415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dff6ed-4593-4eb1-8754-509f235c4fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.377702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dff6ed-4593-4eb1-8754-509f235c4fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.377979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dff6ed-4593-4eb1-8754-509f235c4fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.378676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dff6ed-4593-4eb1-8754-509f235c4fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:41.380946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dff6ed-4593-4eb1-8754-509f235c4fd6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:41.412163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1042/2094] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:41 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:41.413577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f8062-cf18-473b-b8a5-2aa15872a848 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:41.424357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f8062-cf18-473b-b8a5-2aa15872a848 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:41.493188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03180db0-eab0-41f4-ad50-c103cdb8d41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:41.494490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03180db0-eab0-41f4-ad50-c103cdb8d41d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.494490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03180db0-eab0-41f4-ad50-c103cdb8d41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.494490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03180db0-eab0-41f4-ad50-c103cdb8d41d admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.495391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03180db0-eab0-41f4-ad50-c103cdb8d41d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:41.497424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03180db0-eab0-41f4-ad50-c103cdb8d41d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:41.520355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1053/2095] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:41 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:41.524874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dff6ed-4593-4eb1-8754-509f235c4fd6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:41.535991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dff6ed-4593-4eb1-8754-509f235c4fd6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:41.542306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd79c4-5b67-4572-b37a-3afb5a1e8f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:41.543087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd79c4-5b67-4572-b37a-3afb5a1e8f97 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.543629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd79c4-5b67-4572-b37a-3afb5a1e8f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.543965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd79c4-5b67-4572-b37a-3afb5a1e8f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.545410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd79c4-5b67-4572-b37a-3afb5a1e8f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:41.547182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd79c4-5b67-4572-b37a-3afb5a1e8f97 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:41.562892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1043/2096] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:41.564704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03180db0-eab0-41f4-ad50-c103cdb8d41d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:41.574562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03180db0-eab0-41f4-ad50-c103cdb8d41d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:41.579499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35d4bf-979b-48dc-af4f-3545c14c6372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:41.580199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35d4bf-979b-48dc-af4f-3545c14c6372 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.580483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35d4bf-979b-48dc-af4f-3545c14c6372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.580703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35d4bf-979b-48dc-af4f-3545c14c6372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.581424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35d4bf-979b-48dc-af4f-3545c14c6372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:41.583059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb35d4bf-979b-48dc-af4f-3545c14c6372 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:41.594672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1054/2097] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:41.597499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd79c4-5b67-4572-b37a-3afb5a1e8f97 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:41.609034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd79c4-5b67-4572-b37a-3afb5a1e8f97 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:41.616345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c39c3-fc60-45c1-9615-b31c3ed315ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:41.618045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c39c3-fc60-45c1-9615-b31c3ed315ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.618335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c39c3-fc60-45c1-9615-b31c3ed315ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.618625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c39c3-fc60-45c1-9615-b31c3ed315ef admin admin] PATH_INFO: `/v3/projects/1b08cbf67f3f4b4a8fd420ece6024658/users/7bb43dd24ce7468982220fff640b021b/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.642749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c39c3-fc60-45c1-9615-b31c3ed315ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b08cbf67f3f4b4a8fd420ece6024658, user_id=7bb43dd24ce7468982220fff640b021b, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:41.645861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c39c3-fc60-45c1-9615-b31c3ed315ef admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:41.662854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1044/2098] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:41 2022] PUT /identity/v3/projects/1b08cbf67f3f4b4a8fd420ece6024658/users/7bb43dd24ce7468982220fff640b021b/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:41.673809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02084e5-aeb1-429d-ba7f-f66c9f921ffb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.674100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02084e5-aeb1-429d-ba7f-f66c9f921ffb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.674380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02084e5-aeb1-429d-ba7f-f66c9f921ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.705238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f02084e5-aeb1-429d-ba7f-f66c9f921ffb None None] MFA Rules not processed for user `7bb43dd24ce7468982220fff640b021b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:41.706692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f02084e5-aeb1-429d-ba7f-f66c9f921ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:41.781241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1055/2099] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:41.785778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c39c3-fc60-45c1-9615-b31c3ed315ef admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:41.907985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c39c3-fc60-45c1-9615-b31c3ed315ef admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:41.914223 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074e8ed-ca6d-4baf-8bd8-8eecce354d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:41.914959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e8ed-ca6d-4baf-8bd8-8eecce354d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:41.915245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e8ed-ca6d-4baf-8bd8-8eecce354d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:41.915622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074e8ed-ca6d-4baf-8bd8-8eecce354d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:41.916330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074e8ed-ca6d-4baf-8bd8-8eecce354d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:41.918972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074e8ed-ca6d-4baf-8bd8-8eecce354d24 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:41.959841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1045/2100] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:41 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:41.964035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02084e5-aeb1-429d-ba7f-f66c9f921ffb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:41.976488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02084e5-aeb1-429d-ba7f-f66c9f921ffb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:42.021713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:42.047672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cde0e3-a217-4c26-b23c-49165d9d819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:42.048492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cde0e3-a217-4c26-b23c-49165d9d819a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:42.048970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cde0e3-a217-4c26-b23c-49165d9d819a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:42.049323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cde0e3-a217-4c26-b23c-49165d9d819a admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:42.050098 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cde0e3-a217-4c26-b23c-49165d9d819a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:42.052287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cde0e3-a217-4c26-b23c-49165d9d819a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:42.081524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1056/2101] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:41 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:42.085707 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074e8ed-ca6d-4baf-8bd8-8eecce354d24 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:42.096963 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074e8ed-ca6d-4baf-8bd8-8eecce354d24 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:42.102055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8f033-302c-455e-9a40-798951b53057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:42.102731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8f033-302c-455e-9a40-798951b53057 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:42.103004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8f033-302c-455e-9a40-798951b53057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:42.103265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8f033-302c-455e-9a40-798951b53057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:42.104523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8f033-302c-455e-9a40-798951b53057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:42.114193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8f033-302c-455e-9a40-798951b53057 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:42.132498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1046/2102] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:42.139326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cde0e3-a217-4c26-b23c-49165d9d819a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:42.158611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cde0e3-a217-4c26-b23c-49165d9d819a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:42.171320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591580f-c7a2-4b33-83c8-f83a2530b3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:42.172272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591580f-c7a2-4b33-83c8-f83a2530b3ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:42.172555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591580f-c7a2-4b33-83c8-f83a2530b3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:42.172833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591580f-c7a2-4b33-83c8-f83a2530b3ff admin admin] PATH_INFO: `/v3/projects/6ce1d32777d746a8a1108c1c252265e5/users/3ab54dfa1d9e4cc19d5d0881892a55f3/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:42.192317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591580f-c7a2-4b33-83c8-f83a2530b3ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce1d32777d746a8a1108c1c252265e5, user_id=3ab54dfa1d9e4cc19d5d0881892a55f3, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:42.195217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591580f-c7a2-4b33-83c8-f83a2530b3ff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:42.217928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1057/2103] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:42 2022] PUT /identity/v3/projects/6ce1d32777d746a8a1108c1c252265e5/users/3ab54dfa1d9e4cc19d5d0881892a55f3/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:42.221759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8f033-302c-455e-9a40-798951b53057 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:42.247473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8f033-302c-455e-9a40-798951b53057 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:42.326105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:42.360235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f06ce8-5aca-4b51-a5cc-be68f7d7a47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:42.361038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f06ce8-5aca-4b51-a5cc-be68f7d7a47d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:42.361390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f06ce8-5aca-4b51-a5cc-be68f7d7a47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:42.361735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f06ce8-5aca-4b51-a5cc-be68f7d7a47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:42.362773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f06ce8-5aca-4b51-a5cc-be68f7d7a47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:42.364499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f06ce8-5aca-4b51-a5cc-be68f7d7a47d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:42.379164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1047/2104] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:42.380796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591580f-c7a2-4b33-83c8-f83a2530b3ff admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:42.392304 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591580f-c7a2-4b33-83c8-f83a2530b3ff admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:42.396935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c95f4d-37bb-4b37-b3db-54b93e0cae1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:42.397977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c95f4d-37bb-4b37-b3db-54b93e0cae1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:42.398324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c95f4d-37bb-4b37-b3db-54b93e0cae1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:42.398671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c95f4d-37bb-4b37-b3db-54b93e0cae1b admin admin] PATH_INFO: `/v3/projects/6ce1d32777d746a8a1108c1c252265e5/users/3ab54dfa1d9e4cc19d5d0881892a55f3/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:42.401059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c95f4d-37bb-4b37-b3db-54b93e0cae1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce1d32777d746a8a1108c1c252265e5, user_id=3ab54dfa1d9e4cc19d5d0881892a55f3, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:42.403616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c95f4d-37bb-4b37-b3db-54b93e0cae1b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:42.416897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1058/2105] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:42 2022] PUT /identity/v3/projects/6ce1d32777d746a8a1108c1c252265e5/users/3ab54dfa1d9e4cc19d5d0881892a55f3/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:42.431310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20717937-fa76-43e2-9c50-6daff2e2ed9b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:42.431620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20717937-fa76-43e2-9c50-6daff2e2ed9b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:42.431887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20717937-fa76-43e2-9c50-6daff2e2ed9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:42.474613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-20717937-fa76-43e2-9c50-6daff2e2ed9b None None] MFA Rules not processed for user `3ab54dfa1d9e4cc19d5d0881892a55f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:42.475978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-20717937-fa76-43e2-9c50-6daff2e2ed9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:42.558520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c95f4d-37bb-4b37-b3db-54b93e0cae1b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:42.561059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d7c95f4d-37bb-4b37-b3db-54b93e0cae1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:42.608033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1048/2106] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:42.621233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20717937-fa76-43e2-9c50-6daff2e2ed9b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:42.622997 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-20717937-fa76-43e2-9c50-6daff2e2ed9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:42.662199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c95f4d-37bb-4b37-b3db-54b93e0cae1b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:42.671574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:42.672828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:42.673183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:42.673820 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] PATH_INFO: `/v3/users/c58d2efe81f84c498edd6459bbd69be4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:42.676700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58d2efe81f84c498edd6459bbd69be4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:42.680170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:42.706711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20717937-fa76-43e2-9c50-6daff2e2ed9b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:42.717081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59917050-a86e-4820-a9b7-8e0d634ecfb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:42.717895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917050-a86e-4820-a9b7-8e0d634ecfb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:42.718277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917050-a86e-4820-a9b7-8e0d634ecfb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:42.718852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917050-a86e-4820-a9b7-8e0d634ecfb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:42.732599 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59917050-a86e-4820-a9b7-8e0d634ecfb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:42.734153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59917050-a86e-4820-a9b7-8e0d634ecfb8 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:42.746659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58d2efe81f84c498edd6459bbd69be4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:42.747663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58d2efe81f84c498edd6459bbd69be4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:42.747845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1049/2107] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:35:42.759124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58d2efe81f84c498edd6459bbd69be4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:42.781319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58d2efe81f84c498edd6459bbd69be4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:42.782226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58d2efe81f84c498edd6459bbd69be4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:42.802193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1059/2108] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:42 2022] DELETE /identity/v3/users/c58d2efe81f84c498edd6459bbd69be4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:42.842829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59917050-a86e-4820-a9b7-8e0d634ecfb8 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:42.854081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59917050-a86e-4820-a9b7-8e0d634ecfb8 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:42.917854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb1b99-217d-45e8-abeb-045ad0dcebee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:42.919435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb1b99-217d-45e8-abeb-045ad0dcebee service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:42.920197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb1b99-217d-45e8-abeb-045ad0dcebee service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:42.920645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb1b99-217d-45e8-abeb-045ad0dcebee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:42.922517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c7bb1b99-217d-45e8-abeb-045ad0dcebee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:42.942360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:43.007952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb1b99-217d-45e8-abeb-045ad0dcebee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:43.009518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb1b99-217d-45e8-abeb-045ad0dcebee service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:43.019935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1050/2109] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:43.329529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:43.331091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:43.356437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:43.468654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b5bd-112d-45ec-9e59-2d13c7b97e86 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:43.480633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:43.481802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:43.482203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:43.482579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] PATH_INFO: `/v3/projects/1dde0d2953bb48718342af5b9630b178` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:43.484365 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dde0d2953bb48718342af5b9630b178)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:43.490243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:43.566694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dde0d2953bb48718342af5b9630b178', 'request_id': 'req-377788fa-b468-4f2a-8f03-6d75083ae7f4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.573969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dde0d2953bb48718342af5b9630b178', 'request_id': 'req-377788fa-b468-4f2a-8f03-6d75083ae7f4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.582312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dde0d2953bb48718342af5b9630b178', 'request_id': 'req-377788fa-b468-4f2a-8f03-6d75083ae7f4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.606940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] The token cache is being invalidate because project 1dde0d2953bb48718342af5b9630b178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:43.607477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.609322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.611898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1060/2110] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:43 2022] DELETE /identity/v3/projects/1dde0d2953bb48718342af5b9630b178 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:43.616896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb1b99-217d-45e8-abeb-045ad0dcebee service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:43.629012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb1b99-217d-45e8-abeb-045ad0dcebee service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:43.695085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:43.696007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:43.696288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:43.696572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] PATH_INFO: `/v3/users/0bae46342cef432bbb672b168bbf8edb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:43.697770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bae46342cef432bbb672b168bbf8edb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:43.699799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:43.753883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bae46342cef432bbb672b168bbf8edb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.754703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bae46342cef432bbb672b168bbf8edb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.764533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bae46342cef432bbb672b168bbf8edb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.765674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bae46342cef432bbb672b168bbf8edb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.783518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-30294bf4-95a2-4c49-9c0b-3739d45a4fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bae46342cef432bbb672b168bbf8edb'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:43.805120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1051/2111] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:43 2022] DELETE /identity/v3/users/0bae46342cef432bbb672b168bbf8edb => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:44.197849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:44.200953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:44.219838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377788fa-b468-4f2a-8f03-6d75083ae7f4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:44.327842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:44.328890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:44.329174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:44.329451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] PATH_INFO: `/v3/projects/df3cbd2bd8e94817ba1584161b33d101` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:44.334820 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3cbd2bd8e94817ba1584161b33d101)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:44.336847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:44.424806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:44.434134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3cbd2bd8e94817ba1584161b33d101', 'request_id': 'req-5fb44fb9-d3be-4755-adf8-beaa5823a336'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:44.446359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3cbd2bd8e94817ba1584161b33d101', 'request_id': 'req-5fb44fb9-d3be-4755-adf8-beaa5823a336'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:44.458220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3cbd2bd8e94817ba1584161b33d101', 'request_id': 'req-5fb44fb9-d3be-4755-adf8-beaa5823a336'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:44.477870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] The token cache is being invalidate because project df3cbd2bd8e94817ba1584161b33d101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:44.478546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:44.480154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fb44fb9-d3be-4755-adf8-beaa5823a336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:44.482517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1061/2112] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:44 2022] DELETE /identity/v3/projects/df3cbd2bd8e94817ba1584161b33d101 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:44.488000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d0061-8754-46f7-8de5-98a05f648eaf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:44.488496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d0061-8754-46f7-8de5-98a05f648eaf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:44.488731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d0061-8754-46f7-8de5-98a05f648eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:44.510338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-903d0061-8754-46f7-8de5-98a05f648eaf None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:44.511588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-903d0061-8754-46f7-8de5-98a05f648eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:44.556672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:44.610372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1052/2113] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:44.618956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b11a-7e74-4562-a0bd-3977ee89e52d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:44.622631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b11a-7e74-4562-a0bd-3977ee89e52d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:44.623182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b11a-7e74-4562-a0bd-3977ee89e52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:44.654043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-50e3b11a-7e74-4562-a0bd-3977ee89e52d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:44.656327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-50e3b11a-7e74-4562-a0bd-3977ee89e52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:44.704549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1062/2114] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:44.709460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d0061-8754-46f7-8de5-98a05f648eaf None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:44.733052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d0061-8754-46f7-8de5-98a05f648eaf None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:44.751924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8489d22-e226-4835-8cb9-6bec6ac75116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:44.753020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8489d22-e226-4835-8cb9-6bec6ac75116 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:44.753508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8489d22-e226-4835-8cb9-6bec6ac75116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:44.755257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8489d22-e226-4835-8cb9-6bec6ac75116 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:44.756423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8489d22-e226-4835-8cb9-6bec6ac75116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:44.758225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8489d22-e226-4835-8cb9-6bec6ac75116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:44.763180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8489d22-e226-4835-8cb9-6bec6ac75116 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:44.787020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1053/2115] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:44.788796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b11a-7e74-4562-a0bd-3977ee89e52d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:44.800775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b11a-7e74-4562-a0bd-3977ee89e52d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:44.816203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a23f-a68e-494c-8eab-8d707c21f9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:44.817168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a23f-a68e-494c-8eab-8d707c21f9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:44.817683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a23f-a68e-494c-8eab-8d707c21f9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:44.818967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a23f-a68e-494c-8eab-8d707c21f9e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:44.820081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182a23f-a68e-494c-8eab-8d707c21f9e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:44.824702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182a23f-a68e-494c-8eab-8d707c21f9e4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:44.874951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1063/2116] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:44 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:44.884610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8489d22-e226-4835-8cb9-6bec6ac75116 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:44.902406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8489d22-e226-4835-8cb9-6bec6ac75116 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:44.986530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ff300-7e7d-4224-9686-07fc9de54aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:44.987447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ff300-7e7d-4224-9686-07fc9de54aff admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:44.988478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ff300-7e7d-4224-9686-07fc9de54aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:44.988478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ff300-7e7d-4224-9686-07fc9de54aff admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:44.991852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ff300-7e7d-4224-9686-07fc9de54aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:44.994321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ff300-7e7d-4224-9686-07fc9de54aff admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:45.023247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a23f-a68e-494c-8eab-8d707c21f9e4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:45.031545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1054/2117] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:44 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:45.037789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a23f-a68e-494c-8eab-8d707c21f9e4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:45.042774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a8aae-7d12-45ab-9570-b7dfcaac5be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:45.043618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a8aae-7d12-45ab-9570-b7dfcaac5be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.044002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a8aae-7d12-45ab-9570-b7dfcaac5be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.044378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a8aae-7d12-45ab-9570-b7dfcaac5be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.045551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a8aae-7d12-45ab-9570-b7dfcaac5be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:45.047324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a8aae-7d12-45ab-9570-b7dfcaac5be5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:45.060431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1064/2118] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:45.063898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ff300-7e7d-4224-9686-07fc9de54aff admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:45.078731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ff300-7e7d-4224-9686-07fc9de54aff admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:45.085637 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307e607-4826-424e-bba2-9f3e4f07cdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:45.086402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307e607-4826-424e-bba2-9f3e4f07cdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.086766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307e607-4826-424e-bba2-9f3e4f07cdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.087125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307e607-4826-424e-bba2-9f3e4f07cdd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.088005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307e607-4826-424e-bba2-9f3e4f07cdd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:45.089707 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307e607-4826-424e-bba2-9f3e4f07cdd5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:45.101477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1055/2119] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:35:45.105511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a8aae-7d12-45ab-9570-b7dfcaac5be5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:45.117267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a8aae-7d12-45ab-9570-b7dfcaac5be5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:45.122659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb58bf1-f2dd-44eb-80f9-5129ce976f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:45.123695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb58bf1-f2dd-44eb-80f9-5129ce976f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.124046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb58bf1-f2dd-44eb-80f9-5129ce976f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.124390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb58bf1-f2dd-44eb-80f9-5129ce976f4e admin admin] PATH_INFO: `/v3/projects/d4cc6e7692ee40c08e23cc8ed3af4b05/users/9a8aee47a9374451a772cca92c049e7a/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.147036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb58bf1-f2dd-44eb-80f9-5129ce976f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4cc6e7692ee40c08e23cc8ed3af4b05, user_id=9a8aee47a9374451a772cca92c049e7a, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:45.150183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb58bf1-f2dd-44eb-80f9-5129ce976f4e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:45.184226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1cf0a-aa91-47f9-ba53-cf415c54020a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.184226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1cf0a-aa91-47f9-ba53-cf415c54020a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.184226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1cf0a-aa91-47f9-ba53-cf415c54020a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.184920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1065/2120] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:45 2022] PUT /identity/v3/projects/d4cc6e7692ee40c08e23cc8ed3af4b05/users/9a8aee47a9374451a772cca92c049e7a/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:45.215177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6fc1cf0a-aa91-47f9-ba53-cf415c54020a None None] MFA Rules not processed for user `9a8aee47a9374451a772cca92c049e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:45.216666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6fc1cf0a-aa91-47f9-ba53-cf415c54020a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:45.344594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1056/2121] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:45.349273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb58bf1-f2dd-44eb-80f9-5129ce976f4e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:45.438410 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb58bf1-f2dd-44eb-80f9-5129ce976f4e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:45.443413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889f68-f54b-4d1c-855d-e492eae42609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:45.444046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889f68-f54b-4d1c-855d-e492eae42609 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.444323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889f68-f54b-4d1c-855d-e492eae42609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.444595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889f68-f54b-4d1c-855d-e492eae42609 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.445280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8889f68-f54b-4d1c-855d-e492eae42609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:45.447233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8889f68-f54b-4d1c-855d-e492eae42609 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:45.479978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1066/2122] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:45 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:45.484767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1cf0a-aa91-47f9-ba53-cf415c54020a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:45.497769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1cf0a-aa91-47f9-ba53-cf415c54020a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:45.529483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:45.562444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154be1a-3d8a-4224-b7d7-a7b13a599947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:45.563116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154be1a-3d8a-4224-b7d7-a7b13a599947 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.563435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154be1a-3d8a-4224-b7d7-a7b13a599947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.563711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154be1a-3d8a-4224-b7d7-a7b13a599947 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.564404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154be1a-3d8a-4224-b7d7-a7b13a599947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:45.566551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154be1a-3d8a-4224-b7d7-a7b13a599947 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:45.587772 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1057/2123] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:45 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:45.592053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889f68-f54b-4d1c-855d-e492eae42609 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:45.602930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889f68-f54b-4d1c-855d-e492eae42609 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:45.607982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d520a-0c8a-4002-bec4-7480881cb29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:45.608658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d520a-0c8a-4002-bec4-7480881cb29d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.608945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d520a-0c8a-4002-bec4-7480881cb29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.609226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d520a-0c8a-4002-bec4-7480881cb29d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.609969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d520a-0c8a-4002-bec4-7480881cb29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:45.611635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d520a-0c8a-4002-bec4-7480881cb29d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:45.623116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1067/2124] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:45.627261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154be1a-3d8a-4224-b7d7-a7b13a599947 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:45.639782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154be1a-3d8a-4224-b7d7-a7b13a599947 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:45.645828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dcb3d-628b-43e4-86e3-978b2539e693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:45.646493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dcb3d-628b-43e4-86e3-978b2539e693 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.646801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dcb3d-628b-43e4-86e3-978b2539e693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.647049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dcb3d-628b-43e4-86e3-978b2539e693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.647822 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2dcb3d-628b-43e4-86e3-978b2539e693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:45.649439 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2dcb3d-628b-43e4-86e3-978b2539e693 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:45.660254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1058/2125] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:35:45.665282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d520a-0c8a-4002-bec4-7480881cb29d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:45.676273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d520a-0c8a-4002-bec4-7480881cb29d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:45.681463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243cbd0-b8d0-4ce7-ba66-1052652c4ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:45.682333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243cbd0-b8d0-4ce7-ba66-1052652c4ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.682636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243cbd0-b8d0-4ce7-ba66-1052652c4ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.682907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243cbd0-b8d0-4ce7-ba66-1052652c4ae9 admin admin] PATH_INFO: `/v3/projects/ca93a8f29f424d239753c64177f75057/users/9ba269ba5b3a4cf199a91172345821e5/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.698693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243cbd0-b8d0-4ce7-ba66-1052652c4ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca93a8f29f424d239753c64177f75057, user_id=9ba269ba5b3a4cf199a91172345821e5, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:45.701445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243cbd0-b8d0-4ce7-ba66-1052652c4ae9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:45.716050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1068/2126] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:45 2022] PUT /identity/v3/projects/ca93a8f29f424d239753c64177f75057/users/9ba269ba5b3a4cf199a91172345821e5/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:45.787133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a35c8-474d-4063-b530-f0d93110dfed None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.787416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a35c8-474d-4063-b530-f0d93110dfed None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.787723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a35c8-474d-4063-b530-f0d93110dfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.820881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-286a35c8-474d-4063-b530-f0d93110dfed None None] MFA Rules not processed for user `9ba269ba5b3a4cf199a91172345821e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:45.822439 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-286a35c8-474d-4063-b530-f0d93110dfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:45.896195 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1059/2127] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:45.897328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243cbd0-b8d0-4ce7-ba66-1052652c4ae9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:45.922261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:45.972841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243cbd0-b8d0-4ce7-ba66-1052652c4ae9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:45.978463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a707f-43e6-4468-ba05-7a1ed9aa3bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:45.979205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a707f-43e6-4468-ba05-7a1ed9aa3bfb admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:45.979591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a707f-43e6-4468-ba05-7a1ed9aa3bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:45.979939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a707f-43e6-4468-ba05-7a1ed9aa3bfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:45.980788 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a707f-43e6-4468-ba05-7a1ed9aa3bfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:45.982941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a707f-43e6-4468-ba05-7a1ed9aa3bfb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.009483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1069/2128] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:45 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:46.013927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a35c8-474d-4063-b530-f0d93110dfed None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:46.020731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a707f-43e6-4468-ba05-7a1ed9aa3bfb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:46.024330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-736a707f-43e6-4468-ba05-7a1ed9aa3bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:46.029740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a35c8-474d-4063-b530-f0d93110dfed None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:46.093286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a707f-43e6-4468-ba05-7a1ed9aa3bfb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:46.093557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ec07c-f814-45c2-927b-e77009179068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:46.094047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ec07c-f814-45c2-927b-e77009179068 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.094332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ec07c-f814-45c2-927b-e77009179068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.094601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ec07c-f814-45c2-927b-e77009179068 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.095287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ec07c-f814-45c2-927b-e77009179068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:46.097243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ec07c-f814-45c2-927b-e77009179068 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.099448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:46.100714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.101222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.101511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] PATH_INFO: `/v3/users/7bb43dd24ce7468982220fff640b021b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.103114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bb43dd24ce7468982220fff640b021b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:46.105593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.123809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1060/2129] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:46 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:46.124798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ec07c-f814-45c2-927b-e77009179068 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:46.134655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ec07c-f814-45c2-927b-e77009179068 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:46.140528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b6e75-bda0-40b3-96cc-7963a5e85943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:46.141188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b6e75-bda0-40b3-96cc-7963a5e85943 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.141473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b6e75-bda0-40b3-96cc-7963a5e85943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.141757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b6e75-bda0-40b3-96cc-7963a5e85943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.142494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b6e75-bda0-40b3-96cc-7963a5e85943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:46.144168 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b6e75-bda0-40b3-96cc-7963a5e85943 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.154765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1061/2130] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:35:46.159214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b6e75-bda0-40b3-96cc-7963a5e85943 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:46.164512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bb43dd24ce7468982220fff640b021b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:46.166102 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bb43dd24ce7468982220fff640b021b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:46.172632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b6e75-bda0-40b3-96cc-7963a5e85943 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:46.177792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639142d-bc48-4bad-b453-fcd6f75c6756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:46.178417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bb43dd24ce7468982220fff640b021b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:46.178675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639142d-bc48-4bad-b453-fcd6f75c6756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.178964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639142d-bc48-4bad-b453-fcd6f75c6756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.179238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639142d-bc48-4bad-b453-fcd6f75c6756 admin admin] PATH_INFO: `/v3/projects/a5ca155944a745b5bc65ab68ece1729e/users/5ce268bf4d8540068490e5c1a7a41175/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.195779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bb43dd24ce7468982220fff640b021b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:46.196127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639142d-bc48-4bad-b453-fcd6f75c6756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ca155944a745b5bc65ab68ece1729e, user_id=5ce268bf4d8540068490e5c1a7a41175, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:46.196637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bb43dd24ce7468982220fff640b021b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:46.198927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639142d-bc48-4bad-b453-fcd6f75c6756 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.201280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:46.213552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1070/2131] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:46 2022] DELETE /identity/v3/users/7bb43dd24ce7468982220fff640b021b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:46.215621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1062/2132] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:46 2022] PUT /identity/v3/projects/a5ca155944a745b5bc65ab68ece1729e/users/5ce268bf4d8540068490e5c1a7a41175/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:46.219323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:46.219812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631408a9-81f5-4cdd-9352-cb41f5d7a9a1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.220106 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631408a9-81f5-4cdd-9352-cb41f5d7a9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.220413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631408a9-81f5-4cdd-9352-cb41f5d7a9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.220697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:46.237325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-631408a9-81f5-4cdd-9352-cb41f5d7a9a1 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:46.238526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-631408a9-81f5-4cdd-9352-cb41f5d7a9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:46.249210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:46.291473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef10b4b-c6fd-41a8-a5fb-4b5b30b1bed4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:46.296430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32381724-d221-4947-ba92-a4f10750a2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:46.297092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32381724-d221-4947-ba92-a4f10750a2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.297373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32381724-d221-4947-ba92-a4f10750a2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.297646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32381724-d221-4947-ba92-a4f10750a2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.298367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32381724-d221-4947-ba92-a4f10750a2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:46.300021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32381724-d221-4947-ba92-a4f10750a2b4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.303840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1063/2133] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:46.313277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1071/2134] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:35:46.314450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631408a9-81f5-4cdd-9352-cb41f5d7a9a1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:46.315795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32381724-d221-4947-ba92-a4f10750a2b4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:46.317833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-32381724-d221-4947-ba92-a4f10750a2b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:46.324654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631408a9-81f5-4cdd-9352-cb41f5d7a9a1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:46.330076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf217ca-2f56-4bde-9135-3d24269f5b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:46.331077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf217ca-2f56-4bde-9135-3d24269f5b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.331400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf217ca-2f56-4bde-9135-3d24269f5b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.331678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf217ca-2f56-4bde-9135-3d24269f5b68 admin admin] PATH_INFO: `/v3/projects/a5ca155944a745b5bc65ab68ece1729e/users/5ce268bf4d8540068490e5c1a7a41175/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.333695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf217ca-2f56-4bde-9135-3d24269f5b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ca155944a745b5bc65ab68ece1729e, user_id=5ce268bf4d8540068490e5c1a7a41175, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:46.336504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf217ca-2f56-4bde-9135-3d24269f5b68 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.347078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1064/2135] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:46 2022] PUT /identity/v3/projects/a5ca155944a745b5bc65ab68ece1729e/users/5ce268bf4d8540068490e5c1a7a41175/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:46.357942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147dbe9-ff3a-4ff9-b55a-209c662bdf6e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.358333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147dbe9-ff3a-4ff9-b55a-209c662bdf6e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.358705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147dbe9-ff3a-4ff9-b55a-209c662bdf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.378429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32381724-d221-4947-ba92-a4f10750a2b4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:46.385225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6147dbe9-ff3a-4ff9-b55a-209c662bdf6e None None] MFA Rules not processed for user `5ce268bf4d8540068490e5c1a7a41175`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:46.386435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6147dbe9-ff3a-4ff9-b55a-209c662bdf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:46.430924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19307f7-22d1-4a8c-a6b7-459f88afa239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:46.431698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19307f7-22d1-4a8c-a6b7-459f88afa239 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.432044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19307f7-22d1-4a8c-a6b7-459f88afa239 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.432386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19307f7-22d1-4a8c-a6b7-459f88afa239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.469654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1065/2136] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:46.484390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147dbe9-ff3a-4ff9-b55a-209c662bdf6e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:46.485911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6147dbe9-ff3a-4ff9-b55a-209c662bdf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:46.510807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19307f7-22d1-4a8c-a6b7-459f88afa239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:46.512296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19307f7-22d1-4a8c-a6b7-459f88afa239 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.523518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1072/2137] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:46.548855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147dbe9-ff3a-4ff9-b55a-209c662bdf6e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:46.552937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa781f4b-2885-490c-8ce3-1819c7b5c2c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:46.553663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa781f4b-2885-490c-8ce3-1819c7b5c2c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.554023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa781f4b-2885-490c-8ce3-1819c7b5c2c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.554364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa781f4b-2885-490c-8ce3-1819c7b5c2c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.556577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fa781f4b-2885-490c-8ce3-1819c7b5c2c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:46.582240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:46.638191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa781f4b-2885-490c-8ce3-1819c7b5c2c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:46.641053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa781f4b-2885-490c-8ce3-1819c7b5c2c1 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.649904 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1066/2138] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:35:46.920974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19307f7-22d1-4a8c-a6b7-459f88afa239 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:46.922658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d19307f7-22d1-4a8c-a6b7-459f88afa239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:46.947485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19307f7-22d1-4a8c-a6b7-459f88afa239 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:46.960116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:46.961633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:46.962129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:46.962705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] PATH_INFO: `/v3/projects/1b08cbf67f3f4b4a8fd420ece6024658` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:46.965414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b08cbf67f3f4b4a8fd420ece6024658)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:46.967640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:46.994582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:47.034416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b08cbf67f3f4b4a8fd420ece6024658', 'request_id': 'req-a7d4695f-c00e-420e-a261-ca09a8ceb27f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.055769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b08cbf67f3f4b4a8fd420ece6024658', 'request_id': 'req-a7d4695f-c00e-420e-a261-ca09a8ceb27f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.055769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b08cbf67f3f4b4a8fd420ece6024658', 'request_id': 'req-a7d4695f-c00e-420e-a261-ca09a8ceb27f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.072839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] The token cache is being invalidate because project 1b08cbf67f3f4b4a8fd420ece6024658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:47.073421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.074292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.079121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1073/2139] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:46 2022] DELETE /identity/v3/projects/1b08cbf67f3f4b4a8fd420ece6024658 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:47.080561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa781f4b-2885-490c-8ce3-1819c7b5c2c1 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:47.092827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa781f4b-2885-490c-8ce3-1819c7b5c2c1 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:47.163447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:47.164336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:47.164627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:47.164906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] PATH_INFO: `/v3/users/3ab54dfa1d9e4cc19d5d0881892a55f3` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:47.166103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab54dfa1d9e4cc19d5d0881892a55f3)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:47.168118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:47.216942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab54dfa1d9e4cc19d5d0881892a55f3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.217588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab54dfa1d9e4cc19d5d0881892a55f3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.225437 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab54dfa1d9e4cc19d5d0881892a55f3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.226287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab54dfa1d9e4cc19d5d0881892a55f3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.238832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6594138c-a5db-4c03-978e-82b64b1e5a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab54dfa1d9e4cc19d5d0881892a55f3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.252090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1067/2140] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:47 2022] DELETE /identity/v3/users/3ab54dfa1d9e4cc19d5d0881892a55f3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:47.543202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:47.545178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:47.573869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4695f-c00e-420e-a261-ca09a8ceb27f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:47.651129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:47.652039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:47.652363 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:47.652594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] PATH_INFO: `/v3/projects/6ce1d32777d746a8a1108c1c252265e5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:47.653982 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce1d32777d746a8a1108c1c252265e5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:47.656034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:47.718653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce1d32777d746a8a1108c1c252265e5', 'request_id': 'req-419cafc5-196c-4bb8-81df-96e376b8b015'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.733933 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce1d32777d746a8a1108c1c252265e5', 'request_id': 'req-419cafc5-196c-4bb8-81df-96e376b8b015'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.744470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce1d32777d746a8a1108c1c252265e5', 'request_id': 'req-419cafc5-196c-4bb8-81df-96e376b8b015'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.776282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] The token cache is being invalidate because project 6ce1d32777d746a8a1108c1c252265e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:47.776282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.779022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-419cafc5-196c-4bb8-81df-96e376b8b015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:47.781855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1074/2141] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:47 2022] DELETE /identity/v3/projects/6ce1d32777d746a8a1108c1c252265e5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:47.786901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c972d5-19c8-47d2-b4ac-f491e8ed6348 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:47.787519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c972d5-19c8-47d2-b4ac-f491e8ed6348 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:47.787633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c972d5-19c8-47d2-b4ac-f491e8ed6348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:47.806420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c4c972d5-19c8-47d2-b4ac-f491e8ed6348 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:47.807703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c4c972d5-19c8-47d2-b4ac-f491e8ed6348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:47.850493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:47.879900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1068/2142] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:47.888083 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac891d-179e-4fc7-8c8d-8d8455c169a4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:47.888376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac891d-179e-4fc7-8c8d-8d8455c169a4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:47.888659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac891d-179e-4fc7-8c8d-8d8455c169a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:47.908965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-16ac891d-179e-4fc7-8c8d-8d8455c169a4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:47.910224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-16ac891d-179e-4fc7-8c8d-8d8455c169a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:47.919220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1075/2143] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:47.924108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c972d5-19c8-47d2-b4ac-f491e8ed6348 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:47.935187 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c972d5-19c8-47d2-b4ac-f491e8ed6348 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:47.939848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61951d-f448-479e-bb83-4eee29bc487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:47.940518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61951d-f448-479e-bb83-4eee29bc487a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:47.940802 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61951d-f448-479e-bb83-4eee29bc487a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:47.941077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61951d-f448-479e-bb83-4eee29bc487a admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:47.941939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61951d-f448-479e-bb83-4eee29bc487a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:47.942249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61951d-f448-479e-bb83-4eee29bc487a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:47.943985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a61951d-f448-479e-bb83-4eee29bc487a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:47.956734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1069/2144] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:35:47.962019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac891d-179e-4fc7-8c8d-8d8455c169a4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:47.973580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac891d-179e-4fc7-8c8d-8d8455c169a4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:47.978721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47193df-0b9d-4938-9b8d-f6eb43e830ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:47.979493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47193df-0b9d-4938-9b8d-f6eb43e830ec admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:47.979853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47193df-0b9d-4938-9b8d-f6eb43e830ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:47.980332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47193df-0b9d-4938-9b8d-f6eb43e830ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:47.981029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47193df-0b9d-4938-9b8d-f6eb43e830ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:47.983013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47193df-0b9d-4938-9b8d-f6eb43e830ec admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.013842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1076/2145] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:47 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:48.018478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61951d-f448-479e-bb83-4eee29bc487a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.030939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61951d-f448-479e-bb83-4eee29bc487a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.116167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37033d71-f852-44fc-95f8-62192e4f6a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.117543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37033d71-f852-44fc-95f8-62192e4f6a1a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.118227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37033d71-f852-44fc-95f8-62192e4f6a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.118517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37033d71-f852-44fc-95f8-62192e4f6a1a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.119489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37033d71-f852-44fc-95f8-62192e4f6a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.121976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37033d71-f852-44fc-95f8-62192e4f6a1a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.156343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1070/2146] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:48 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:48.157828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47193df-0b9d-4938-9b8d-f6eb43e830ec admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.175150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47193df-0b9d-4938-9b8d-f6eb43e830ec admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.182451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2916a64-ca43-41e9-a564-0523bf315104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.185973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2916a64-ca43-41e9-a564-0523bf315104 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.186252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2916a64-ca43-41e9-a564-0523bf315104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.186795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2916a64-ca43-41e9-a564-0523bf315104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.188707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2916a64-ca43-41e9-a564-0523bf315104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.190732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2916a64-ca43-41e9-a564-0523bf315104 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.197714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:48.207045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1077/2147] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:48.208898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37033d71-f852-44fc-95f8-62192e4f6a1a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.220549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37033d71-f852-44fc-95f8-62192e4f6a1a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.226004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ba0e0-1d2f-4421-952c-0c18c1896908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.226841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ba0e0-1d2f-4421-952c-0c18c1896908 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.227148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ba0e0-1d2f-4421-952c-0c18c1896908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.227517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ba0e0-1d2f-4421-952c-0c18c1896908 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.228437 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ba0e0-1d2f-4421-952c-0c18c1896908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.230987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ba0e0-1d2f-4421-952c-0c18c1896908 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.244428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1071/2148] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:48.248584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2916a64-ca43-41e9-a564-0523bf315104 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.262360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2916a64-ca43-41e9-a564-0523bf315104 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.272577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75619f0d-8d32-454d-b3a5-1a470b6ff5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.273546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75619f0d-8d32-454d-b3a5-1a470b6ff5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.273834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75619f0d-8d32-454d-b3a5-1a470b6ff5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.274252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75619f0d-8d32-454d-b3a5-1a470b6ff5e0 admin admin] PATH_INFO: `/v3/projects/e1214f3cba564373b917e65afbba84e5/users/3330994eed2d4bd0ad31895ca09b96fc/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.305753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75619f0d-8d32-454d-b3a5-1a470b6ff5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1214f3cba564373b917e65afbba84e5, user_id=3330994eed2d4bd0ad31895ca09b96fc, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.308753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75619f0d-8d32-454d-b3a5-1a470b6ff5e0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.331286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1078/2149] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:48 2022] PUT /identity/v3/projects/e1214f3cba564373b917e65afbba84e5/users/3330994eed2d4bd0ad31895ca09b96fc/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:48.340586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ba0e0-1d2f-4421-952c-0c18c1896908 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.343837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-122ba0e0-1d2f-4421-952c-0c18c1896908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:48.344146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafb010-ab32-4b59-a6fe-e4754c70aaf9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.344434 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafb010-ab32-4b59-a6fe-e4754c70aaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.344720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafb010-ab32-4b59-a6fe-e4754c70aaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.386188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-efafb010-ab32-4b59-a6fe-e4754c70aaf9 None None] MFA Rules not processed for user `3330994eed2d4bd0ad31895ca09b96fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:48.387439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-efafb010-ab32-4b59-a6fe-e4754c70aaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:48.425175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ba0e0-1d2f-4421-952c-0c18c1896908 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.462035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1079/2150] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:48.468318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafb010-ab32-4b59-a6fe-e4754c70aaf9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.499600 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b9ba2-7c32-4de1-8d64-e06dfd2a75b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.501163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b9ba2-7c32-4de1-8d64-e06dfd2a75b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.501445 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b9ba2-7c32-4de1-8d64-e06dfd2a75b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.501726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b9ba2-7c32-4de1-8d64-e06dfd2a75b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.504721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-625b9ba2-7c32-4de1-8d64-e06dfd2a75b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:48.519315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:48.561644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafb010-ab32-4b59-a6fe-e4754c70aaf9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.566771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea4850-a7ee-4651-9ec0-9afeeca0715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.567569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea4850-a7ee-4651-9ec0-9afeeca0715a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.567923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea4850-a7ee-4651-9ec0-9afeeca0715a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.568201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea4850-a7ee-4651-9ec0-9afeeca0715a admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.568901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea4850-a7ee-4651-9ec0-9afeeca0715a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.570832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea4850-a7ee-4651-9ec0-9afeeca0715a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.582007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b9ba2-7c32-4de1-8d64-e06dfd2a75b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.583999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b9ba2-7c32-4de1-8d64-e06dfd2a75b7 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.594273 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1072/2151] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:48.602917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1080/2152] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:48 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:48.607105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b9ba2-7c32-4de1-8d64-e06dfd2a75b7 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.618819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b9ba2-7c32-4de1-8d64-e06dfd2a75b7 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.724709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450fba4-011e-48f1-8c9d-a5290e2472a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.725538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450fba4-011e-48f1-8c9d-a5290e2472a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.725890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450fba4-011e-48f1-8c9d-a5290e2472a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.726255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450fba4-011e-48f1-8c9d-a5290e2472a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.727287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450fba4-011e-48f1-8c9d-a5290e2472a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.730154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450fba4-011e-48f1-8c9d-a5290e2472a1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.759127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1073/2153] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:48 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:48.769085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea4850-a7ee-4651-9ec0-9afeeca0715a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.790615 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea4850-a7ee-4651-9ec0-9afeeca0715a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.797853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae777ec5-1487-4bd2-9bed-63c1c3e59ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.799991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae777ec5-1487-4bd2-9bed-63c1c3e59ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.800857 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae777ec5-1487-4bd2-9bed-63c1c3e59ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.801149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae777ec5-1487-4bd2-9bed-63c1c3e59ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.801936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae777ec5-1487-4bd2-9bed-63c1c3e59ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.804370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae777ec5-1487-4bd2-9bed-63c1c3e59ad9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.826995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1081/2154] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:48.831208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450fba4-011e-48f1-8c9d-a5290e2472a1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.845838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450fba4-011e-48f1-8c9d-a5290e2472a1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.854500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21163383-6938-401d-830c-f47ca6bbfaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.855755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21163383-6938-401d-830c-f47ca6bbfaae admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.856202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21163383-6938-401d-830c-f47ca6bbfaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.856619 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21163383-6938-401d-830c-f47ca6bbfaae admin admin] PATH_INFO: `/v3/projects/61ec1975f2574cc09dc51b6533b860a8/users/f607da07b94f491a8332228eabf2de34/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.874435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21163383-6938-401d-830c-f47ca6bbfaae admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ec1975f2574cc09dc51b6533b860a8, user_id=f607da07b94f491a8332228eabf2de34, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.877492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21163383-6938-401d-830c-f47ca6bbfaae admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:48.894056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1074/2155] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:48 2022] PUT /identity/v3/projects/61ec1975f2574cc09dc51b6533b860a8/users/f607da07b94f491a8332228eabf2de34/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:48.899820 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae777ec5-1487-4bd2-9bed-63c1c3e59ad9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:48.913545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae777ec5-1487-4bd2-9bed-63c1c3e59ad9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:48.962796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:48.993424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b245af-b60d-467d-945b-0b5465b5645a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:48.994281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b245af-b60d-467d-945b-0b5465b5645a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:48.994634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b245af-b60d-467d-945b-0b5465b5645a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:48.994956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b245af-b60d-467d-945b-0b5465b5645a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:48.995729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b245af-b60d-467d-945b-0b5465b5645a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:48.997369 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b245af-b60d-467d-945b-0b5465b5645a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:49.009333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1082/2156] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:35:49.013580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21163383-6938-401d-830c-f47ca6bbfaae admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:49.027470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21163383-6938-401d-830c-f47ca6bbfaae admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:49.043758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55398d83-d296-4a30-ac3e-0ee78d270b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:49.045193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55398d83-d296-4a30-ac3e-0ee78d270b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:49.045530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55398d83-d296-4a30-ac3e-0ee78d270b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:49.045873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55398d83-d296-4a30-ac3e-0ee78d270b2e admin admin] PATH_INFO: `/v3/projects/61ec1975f2574cc09dc51b6533b860a8/users/f607da07b94f491a8332228eabf2de34/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:49.052583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55398d83-d296-4a30-ac3e-0ee78d270b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ec1975f2574cc09dc51b6533b860a8, user_id=f607da07b94f491a8332228eabf2de34, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:49.055518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55398d83-d296-4a30-ac3e-0ee78d270b2e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:49.071489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1075/2157] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:49 2022] PUT /identity/v3/projects/61ec1975f2574cc09dc51b6533b860a8/users/f607da07b94f491a8332228eabf2de34/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:49.083173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8ab1b-a178-4b7c-b09d-b2ae5989ebee None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:49.084150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8ab1b-a178-4b7c-b09d-b2ae5989ebee None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:49.085003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8ab1b-a178-4b7c-b09d-b2ae5989ebee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:49.154362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5db8ab1b-a178-4b7c-b09d-b2ae5989ebee None None] MFA Rules not processed for user `f607da07b94f491a8332228eabf2de34`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:49.155749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5db8ab1b-a178-4b7c-b09d-b2ae5989ebee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:49.285357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1083/2158] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:49.297896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55398d83-d296-4a30-ac3e-0ee78d270b2e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:49.363757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55398d83-d296-4a30-ac3e-0ee78d270b2e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:49.386315 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89ea91-3cd1-4230-bfc9-05d84ec7afc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:49.387100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89ea91-3cd1-4230-bfc9-05d84ec7afc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:49.387415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89ea91-3cd1-4230-bfc9-05d84ec7afc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:49.387769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89ea91-3cd1-4230-bfc9-05d84ec7afc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:49.399068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89ea91-3cd1-4230-bfc9-05d84ec7afc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:49.400790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89ea91-3cd1-4230-bfc9-05d84ec7afc3 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:49.413625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1076/2159] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:49.424239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8ab1b-a178-4b7c-b09d-b2ae5989ebee None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:49.425900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5db8ab1b-a178-4b7c-b09d-b2ae5989ebee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:49.448956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8ab1b-a178-4b7c-b09d-b2ae5989ebee None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:49.449350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ffe96-4965-48a3-a5e6-c35bfde13b4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:49.450244 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ffe96-4965-48a3-a5e6-c35bfde13b4b service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:49.450579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ffe96-4965-48a3-a5e6-c35bfde13b4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:49.450834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ffe96-4965-48a3-a5e6-c35bfde13b4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:49.453145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0f6ffe96-4965-48a3-a5e6-c35bfde13b4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:49.485147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89ea91-3cd1-4230-bfc9-05d84ec7afc3 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:49.504850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:49.544048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ffe96-4965-48a3-a5e6-c35bfde13b4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:49.545897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ffe96-4965-48a3-a5e6-c35bfde13b4b service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:49.556600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1084/2160] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:49.566568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89ea91-3cd1-4230-bfc9-05d84ec7afc3 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:49.576232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:49.577127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:49.577409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:49.577683 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] PATH_INFO: `/v3/users/3330994eed2d4bd0ad31895ca09b96fc` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:49.579834 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3330994eed2d4bd0ad31895ca09b96fc)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:49.581807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:49.646281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3330994eed2d4bd0ad31895ca09b96fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:49.646977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3330994eed2d4bd0ad31895ca09b96fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:49.657864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3330994eed2d4bd0ad31895ca09b96fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:49.658569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3330994eed2d4bd0ad31895ca09b96fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:49.672811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3330994eed2d4bd0ad31895ca09b96fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:49.691404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1077/2161] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:49 2022] DELETE /identity/v3/users/3330994eed2d4bd0ad31895ca09b96fc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:49.694901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0180e-8302-4592-a784-ae2062d9ff86 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:49.694901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0180e-8302-4592-a784-ae2062d9ff86 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:49.694901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0180e-8302-4592-a784-ae2062d9ff86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:49.717000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9aa0180e-8302-4592-a784-ae2062d9ff86 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:49.718364 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9aa0180e-8302-4592-a784-ae2062d9ff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:49.794825 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:49.807744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1085/2162] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:49.817882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:49.819527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:49.889393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04118209-ea66-4f91-a16f-fb035bd07d6b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:49.893549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3b9d4-67b9-4ae8-b25d-9e51cab8406f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:49.894512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3b9d4-67b9-4ae8-b25d-9e51cab8406f service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:49.894824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3b9d4-67b9-4ae8-b25d-9e51cab8406f service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:49.895409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3b9d4-67b9-4ae8-b25d-9e51cab8406f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:49.907430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3b9d4-67b9-4ae8-b25d-9e51cab8406f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:49.909096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3b9d4-67b9-4ae8-b25d-9e51cab8406f service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:49.921333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1078/2163] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:50.322989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0180e-8302-4592-a784-ae2062d9ff86 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:50.325230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9aa0180e-8302-4592-a784-ae2062d9ff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:50.354462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0180e-8302-4592-a784-ae2062d9ff86 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:50.360843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:50.362032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:50.362327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:50.362602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] PATH_INFO: `/v3/projects/e1214f3cba564373b917e65afbba84e5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:50.364801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1214f3cba564373b917e65afbba84e5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:50.366935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:50.461400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1214f3cba564373b917e65afbba84e5', 'request_id': 'req-b3d13f53-ca90-46ec-a3e0-e7d73517720a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.471848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1214f3cba564373b917e65afbba84e5', 'request_id': 'req-b3d13f53-ca90-46ec-a3e0-e7d73517720a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.481088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1214f3cba564373b917e65afbba84e5', 'request_id': 'req-b3d13f53-ca90-46ec-a3e0-e7d73517720a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.498371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] The token cache is being invalidate because project e1214f3cba564373b917e65afbba84e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:50.498750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.499361 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.504376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1086/2164] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:50 2022] DELETE /identity/v3/projects/e1214f3cba564373b917e65afbba84e5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:50.505902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3b9d4-67b9-4ae8-b25d-9e51cab8406f service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:50.517418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3b9d4-67b9-4ae8-b25d-9e51cab8406f service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:50.558996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:50.599321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:50.600721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:50.601071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:50.601415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] PATH_INFO: `/v3/users/f607da07b94f491a8332228eabf2de34` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:50.602753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f607da07b94f491a8332228eabf2de34)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:50.604901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:50.662658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f607da07b94f491a8332228eabf2de34'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.663699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f607da07b94f491a8332228eabf2de34'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.690388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f607da07b94f491a8332228eabf2de34'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.691143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f607da07b94f491a8332228eabf2de34'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.710386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f607da07b94f491a8332228eabf2de34'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:50.740445 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1079/2165] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:50 2022] DELETE /identity/v3/users/f607da07b94f491a8332228eabf2de34 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:51.111964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.114512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:51.145077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d13f53-ca90-46ec-a3e0-e7d73517720a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.184878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.187048 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:51.237977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.239117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.239669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.240158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] PATH_INFO: `/v3/users/9028859afbab40df9fc4e5ea6d876931` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.241688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9028859afbab40df9fc4e5ea6d876931)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.244491 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.266843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abed30-cf03-4710-88b8-1f85cd2a76cd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.272389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.273527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.273930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.274538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] PATH_INFO: `/v3/projects/61ec1975f2574cc09dc51b6533b860a8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.276072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ec1975f2574cc09dc51b6533b860a8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.278491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.297210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9028859afbab40df9fc4e5ea6d876931'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.298050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9028859afbab40df9fc4e5ea6d876931'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.306876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9028859afbab40df9fc4e5ea6d876931'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.319030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9028859afbab40df9fc4e5ea6d876931'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.319741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02e3cd27-f5e1-4795-9613-259ed6912e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9028859afbab40df9fc4e5ea6d876931'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.327500 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:51.328071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ec1975f2574cc09dc51b6533b860a8', 'request_id': 'req-c69fe610-4f66-480a-bf80-7f14488ac28a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.332087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:51.332862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1087/2166] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:51 2022] DELETE /identity/v3/users/9028859afbab40df9fc4e5ea6d876931 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:51.337324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ec1975f2574cc09dc51b6533b860a8', 'request_id': 'req-c69fe610-4f66-480a-bf80-7f14488ac28a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.345625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ec1975f2574cc09dc51b6533b860a8', 'request_id': 'req-c69fe610-4f66-480a-bf80-7f14488ac28a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.358054 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] The token cache is being invalidate because project 61ec1975f2574cc09dc51b6533b860a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:51.358341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.359030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c69fe610-4f66-480a-bf80-7f14488ac28a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.363330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1080/2167] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:51 2022] DELETE /identity/v3/projects/61ec1975f2574cc09dc51b6533b860a8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:51.365920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bda19-3757-46b5-8a16-1a181dbaeafd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.366209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bda19-3757-46b5-8a16-1a181dbaeafd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.366492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bda19-3757-46b5-8a16-1a181dbaeafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.385323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a99bda19-3757-46b5-8a16-1a181dbaeafd None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:51.386533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a99bda19-3757-46b5-8a16-1a181dbaeafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:51.449548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1088/2168] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:51.457745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3ba48-1040-4d69-b368-3463faeeb2d1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.458158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3ba48-1040-4d69-b368-3463faeeb2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.458523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3ba48-1040-4d69-b368-3463faeeb2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.477614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c5a3ba48-1040-4d69-b368-3463faeeb2d1 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:51.479097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c5a3ba48-1040-4d69-b368-3463faeeb2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:51.487222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1081/2169] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:51.491922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bda19-3757-46b5-8a16-1a181dbaeafd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.502772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bda19-3757-46b5-8a16-1a181dbaeafd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.508481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f93bc-bec2-4535-b2a0-7bc6e57b4777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.509155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f93bc-bec2-4535-b2a0-7bc6e57b4777 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.509465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f93bc-bec2-4535-b2a0-7bc6e57b4777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.509778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f93bc-bec2-4535-b2a0-7bc6e57b4777 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.510723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f93bc-bec2-4535-b2a0-7bc6e57b4777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:51.511028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f93bc-bec2-4535-b2a0-7bc6e57b4777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.512882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f93bc-bec2-4535-b2a0-7bc6e57b4777 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.524668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3ba48-1040-4d69-b368-3463faeeb2d1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.526399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c5a3ba48-1040-4d69-b368-3463faeeb2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:51.533241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1089/2170] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:35:51.534085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f93bc-bec2-4535-b2a0-7bc6e57b4777 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.540184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3ba48-1040-4d69-b368-3463faeeb2d1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.545403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f93bc-bec2-4535-b2a0-7bc6e57b4777 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.545534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.546390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.546675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.546948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] PATH_INFO: `/v3/projects/dedc7a5a95d14f60b1a73198c8b5309b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.548223 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] RBAC: Authorizing `identity:delete_project(project_id=dedc7a5a95d14f60b1a73198c8b5309b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.550123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.551736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59273f0-ae4b-47f0-9001-0ce36c9cd216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.552803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59273f0-ae4b-47f0-9001-0ce36c9cd216 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.552968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59273f0-ae4b-47f0-9001-0ce36c9cd216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.553266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59273f0-ae4b-47f0-9001-0ce36c9cd216 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.554177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59273f0-ae4b-47f0-9001-0ce36c9cd216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.556759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59273f0-ae4b-47f0-9001-0ce36c9cd216 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.591084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1090/2171] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:51 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:51.592544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59273f0-ae4b-47f0-9001-0ce36c9cd216 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.603950 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dedc7a5a95d14f60b1a73198c8b5309b', 'request_id': 'req-48d51c87-f3e2-49c7-9455-3404ab01d4db'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.612610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dedc7a5a95d14f60b1a73198c8b5309b', 'request_id': 'req-48d51c87-f3e2-49c7-9455-3404ab01d4db'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.621216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dedc7a5a95d14f60b1a73198c8b5309b', 'request_id': 'req-48d51c87-f3e2-49c7-9455-3404ab01d4db'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.632881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] The token cache is being invalidate because project dedc7a5a95d14f60b1a73198c8b5309b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:51.633166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.633824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-48d51c87-f3e2-49c7-9455-3404ab01d4db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:51.635753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1082/2172] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:51 2022] DELETE /identity/v3/projects/dedc7a5a95d14f60b1a73198c8b5309b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:51.641335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d08cc9-9984-4f25-b651-bc10eb51bcd1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.641622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d08cc9-9984-4f25-b651-bc10eb51bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.641908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d08cc9-9984-4f25-b651-bc10eb51bcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.656547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59273f0-ae4b-47f0-9001-0ce36c9cd216 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.661128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-75d08cc9-9984-4f25-b651-bc10eb51bcd1 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:51.663157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-75d08cc9-9984-4f25-b651-bc10eb51bcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:51.715233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d794b-f03e-4779-8c03-56c98d1fb821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.715873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d794b-f03e-4779-8c03-56c98d1fb821 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.716155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d794b-f03e-4779-8c03-56c98d1fb821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.716442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d794b-f03e-4779-8c03-56c98d1fb821 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.717152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d794b-f03e-4779-8c03-56c98d1fb821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.719115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d794b-f03e-4779-8c03-56c98d1fb821 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.725100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1083/2173] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:51.733314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6710d-8a52-4bbb-8479-40fcc038fbce None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.733663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6710d-8a52-4bbb-8479-40fcc038fbce None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.733887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6710d-8a52-4bbb-8479-40fcc038fbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.735556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:51.744414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1091/2174] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:51 2022] POST /identity/v3/users => generated 305 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:51.745449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d794b-f03e-4779-8c03-56c98d1fb821 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.748100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bf3d794b-f03e-4779-8c03-56c98d1fb821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:51.752993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3fb6710d-8a52-4bbb-8479-40fcc038fbce None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:51.754156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3fb6710d-8a52-4bbb-8479-40fcc038fbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:51.760328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d794b-f03e-4779-8c03-56c98d1fb821 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.763002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1084/2175] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:51.766995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f75451-0ba5-4698-b330-304235e84177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.767140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6710d-8a52-4bbb-8479-40fcc038fbce None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.767715 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f75451-0ba5-4698-b330-304235e84177 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.768033 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f75451-0ba5-4698-b330-304235e84177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.768315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f75451-0ba5-4698-b330-304235e84177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.769087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f75451-0ba5-4698-b330-304235e84177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.771024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f75451-0ba5-4698-b330-304235e84177 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.776711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6710d-8a52-4bbb-8479-40fcc038fbce None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.781395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7560c-39fc-4628-a426-505500c6be86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.782035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7560c-39fc-4628-a426-505500c6be86 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.782204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1092/2176] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:51.782393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7560c-39fc-4628-a426-505500c6be86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.782674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7560c-39fc-4628-a426-505500c6be86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.783492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7560c-39fc-4628-a426-505500c6be86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:51.783797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7560c-39fc-4628-a426-505500c6be86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.785552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7560c-39fc-4628-a426-505500c6be86 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.786148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f75451-0ba5-4698-b330-304235e84177 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.797131 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f75451-0ba5-4698-b330-304235e84177 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.801410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1085/2177] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:35:51.802190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd488c7-0515-4049-89bd-71a39ddc5a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.803008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7560c-39fc-4628-a426-505500c6be86 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.803124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd488c7-0515-4049-89bd-71a39ddc5a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.803426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd488c7-0515-4049-89bd-71a39ddc5a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.803706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd488c7-0515-4049-89bd-71a39ddc5a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.804438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd488c7-0515-4049-89bd-71a39ddc5a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.806079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd488c7-0515-4049-89bd-71a39ddc5a5d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.812946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7560c-39fc-4628-a426-505500c6be86 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.817124 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1093/2178] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:51.818462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2321f-f8c8-407f-b151-8f5047ef3ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.819155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2321f-f8c8-407f-b151-8f5047ef3ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.819481 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2321f-f8c8-407f-b151-8f5047ef3ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.819766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2321f-f8c8-407f-b151-8f5047ef3ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.820731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2321f-f8c8-407f-b151-8f5047ef3ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.821239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd488c7-0515-4049-89bd-71a39ddc5a5d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.822656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2321f-f8c8-407f-b151-8f5047ef3ff5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.831278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd488c7-0515-4049-89bd-71a39ddc5a5d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.836679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfa1c8-87ff-48e7-af03-010e2c57b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:51.837797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfa1c8-87ff-48e7-af03-010e2c57b7ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.838181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfa1c8-87ff-48e7-af03-010e2c57b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.838455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfa1c8-87ff-48e7-af03-010e2c57b7ff admin admin] PATH_INFO: `/v3/projects/b0facc0864d7407cbdfac93d17cdee82/users/184e4c13c82e4c57aed348b882284a51/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.854215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1086/2179] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:51 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:51.855246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2321f-f8c8-407f-b151-8f5047ef3ff5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:51.857737 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfa1c8-87ff-48e7-af03-010e2c57b7ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0facc0864d7407cbdfac93d17cdee82, user_id=184e4c13c82e4c57aed348b882284a51, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:51.861594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfa1c8-87ff-48e7-af03-010e2c57b7ff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:51.876206 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:51.892058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1094/2180] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:51 2022] PUT /identity/v3/projects/b0facc0864d7407cbdfac93d17cdee82/users/184e4c13c82e4c57aed348b882284a51/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:51.896555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427c35b-d30b-4fd6-9f7a-60267fd9ab4e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:51.898581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427c35b-d30b-4fd6-9f7a-60267fd9ab4e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:51.898581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427c35b-d30b-4fd6-9f7a-60267fd9ab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:51.926432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1427c35b-d30b-4fd6-9f7a-60267fd9ab4e None None] MFA Rules not processed for user `184e4c13c82e4c57aed348b882284a51`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:51.927728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1427c35b-d30b-4fd6-9f7a-60267fd9ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:51.943163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2321f-f8c8-407f-b151-8f5047ef3ff5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:51.999030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1095/2181] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:52.003502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427c35b-d30b-4fd6-9f7a-60267fd9ab4e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.021880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e648d3-d73e-49e4-b592-730ef3048ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.022656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e648d3-d73e-49e4-b592-730ef3048ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.023212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e648d3-d73e-49e4-b592-730ef3048ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.023395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e648d3-d73e-49e4-b592-730ef3048ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.024151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e648d3-d73e-49e4-b592-730ef3048ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.026166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e648d3-d73e-49e4-b592-730ef3048ac1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.049135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1087/2182] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:51 2022] POST /identity/v3/users => generated 284 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:52.053182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e648d3-d73e-49e4-b592-730ef3048ac1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.063382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e648d3-d73e-49e4-b592-730ef3048ac1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.067945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe2907-0b04-464c-a1c2-307c4da3b14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.068592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe2907-0b04-464c-a1c2-307c4da3b14a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.068872 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe2907-0b04-464c-a1c2-307c4da3b14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.069146 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe2907-0b04-464c-a1c2-307c4da3b14a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.069872 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe2907-0b04-464c-a1c2-307c4da3b14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.071495 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe2907-0b04-464c-a1c2-307c4da3b14a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.075964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427c35b-d30b-4fd6-9f7a-60267fd9ab4e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.080748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f4d8b-ba44-4081-ba47-06a8a90a24d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.081702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1088/2183] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:35:52.081818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f4d8b-ba44-4081-ba47-06a8a90a24d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.082092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f4d8b-ba44-4081-ba47-06a8a90a24d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.082403 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f4d8b-ba44-4081-ba47-06a8a90a24d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.083250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f4d8b-ba44-4081-ba47-06a8a90a24d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.085435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f4d8b-ba44-4081-ba47-06a8a90a24d7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.085544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe2907-0b04-464c-a1c2-307c4da3b14a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.095084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe2907-0b04-464c-a1c2-307c4da3b14a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.099663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf13ce-2898-40fe-a5b9-84406775120d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.100334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf13ce-2898-40fe-a5b9-84406775120d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.100625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf13ce-2898-40fe-a5b9-84406775120d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.100905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf13ce-2898-40fe-a5b9-84406775120d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.101618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf13ce-2898-40fe-a5b9-84406775120d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.103230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf13ce-2898-40fe-a5b9-84406775120d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.112320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1096/2184] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:52 2022] POST /identity/v3/projects => generated 407 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:52.116273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f4d8b-ba44-4081-ba47-06a8a90a24d7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.120496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1089/2185] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:52.125603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf13ce-2898-40fe-a5b9-84406775120d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.135888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf13ce-2898-40fe-a5b9-84406775120d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.147509 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:52.188245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f4d8b-ba44-4081-ba47-06a8a90a24d7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.193091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0214283-e610-484d-b32f-11355418e520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.193940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0214283-e610-484d-b32f-11355418e520 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.194269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0214283-e610-484d-b32f-11355418e520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.194549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0214283-e610-484d-b32f-11355418e520 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.195244 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0214283-e610-484d-b32f-11355418e520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.197216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0214283-e610-484d-b32f-11355418e520 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.199197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15194de6-6377-4157-aca1-8e492a59ef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.200027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15194de6-6377-4157-aca1-8e492a59ef7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.200307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15194de6-6377-4157-aca1-8e492a59ef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.200576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15194de6-6377-4157-aca1-8e492a59ef7f admin admin] PATH_INFO: `/v3/projects/87f3bfb730144616b5e329cf0cfd37ca/users/abeeca8832e54b7ebf1aebc6d1f0e9af/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.218285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15194de6-6377-4157-aca1-8e492a59ef7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f3bfb730144616b5e329cf0cfd37ca, user_id=abeeca8832e54b7ebf1aebc6d1f0e9af, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.220987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15194de6-6377-4157-aca1-8e492a59ef7f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.229113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1097/2186] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:52 2022] POST /identity/v3/users => generated 310 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:52.230210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0214283-e610-484d-b32f-11355418e520 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.236305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1090/2187] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:52 2022] PUT /identity/v3/projects/87f3bfb730144616b5e329cf0cfd37ca/users/abeeca8832e54b7ebf1aebc6d1f0e9af/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:52.245250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0214283-e610-484d-b32f-11355418e520 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.248795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a317d8-f238-46d5-8256-8214c98a8106 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.249088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a317d8-f238-46d5-8256-8214c98a8106 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.249364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a317d8-f238-46d5-8256-8214c98a8106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.278148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e6a317d8-f238-46d5-8256-8214c98a8106 None None] MFA Rules not processed for user `abeeca8832e54b7ebf1aebc6d1f0e9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:52.279460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e6a317d8-f238-46d5-8256-8214c98a8106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:52.283542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:52.303450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802f543-846f-4bf6-9465-8e4b82b5794e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.304133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802f543-846f-4bf6-9465-8e4b82b5794e admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.304411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802f543-846f-4bf6-9465-8e4b82b5794e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.304749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802f543-846f-4bf6-9465-8e4b82b5794e admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.305503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802f543-846f-4bf6-9465-8e4b82b5794e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.307177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802f543-846f-4bf6-9465-8e4b82b5794e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.317175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1098/2188] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:52.321492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802f543-846f-4bf6-9465-8e4b82b5794e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.330835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802f543-846f-4bf6-9465-8e4b82b5794e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.335534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af4b95-b8df-41b1-970f-2cdca136bd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.336394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af4b95-b8df-41b1-970f-2cdca136bd08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.336668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af4b95-b8df-41b1-970f-2cdca136bd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.336942 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af4b95-b8df-41b1-970f-2cdca136bd08 admin admin] PATH_INFO: `/v3/projects/bdebc5336dbb407ebc01513c3e503a94/users/28fefb0a794b43e39e96a0cbac5d1341/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.351714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1091/2189] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:52.352140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af4b95-b8df-41b1-970f-2cdca136bd08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdebc5336dbb407ebc01513c3e503a94, user_id=28fefb0a794b43e39e96a0cbac5d1341, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.354813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af4b95-b8df-41b1-970f-2cdca136bd08 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.364120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a317d8-f238-46d5-8256-8214c98a8106 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.366625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e6a317d8-f238-46d5-8256-8214c98a8106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:52.370977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1099/2190] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:52 2022] PUT /identity/v3/projects/bdebc5336dbb407ebc01513c3e503a94/users/28fefb0a794b43e39e96a0cbac5d1341/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:52.374862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af4b95-b8df-41b1-970f-2cdca136bd08 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.423306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a317d8-f238-46d5-8256-8214c98a8106 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.436451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af4b95-b8df-41b1-970f-2cdca136bd08 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.441173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec67783-1d6a-434f-b444-da7ff6d63783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.441863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec67783-1d6a-434f-b444-da7ff6d63783 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.442147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec67783-1d6a-434f-b444-da7ff6d63783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.442487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec67783-1d6a-434f-b444-da7ff6d63783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.443270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec67783-1d6a-434f-b444-da7ff6d63783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.444920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec67783-1d6a-434f-b444-da7ff6d63783 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.458321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1100/2191] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:35:52.460250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec67783-1d6a-434f-b444-da7ff6d63783 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.469945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec67783-1d6a-434f-b444-da7ff6d63783 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.473523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3023e-d87f-47f7-b3c9-2ff0a0ae2a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.474324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3023e-d87f-47f7-b3c9-2ff0a0ae2a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.474606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3023e-d87f-47f7-b3c9-2ff0a0ae2a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.474879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3023e-d87f-47f7-b3c9-2ff0a0ae2a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.475110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd954cc-3224-4252-9a11-877fe9be3319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.476024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd954cc-3224-4252-9a11-877fe9be3319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.476302 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd954cc-3224-4252-9a11-877fe9be3319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.476712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd954cc-3224-4252-9a11-877fe9be3319 admin admin] PATH_INFO: `/v3/projects/bdebc5336dbb407ebc01513c3e503a94/users/28fefb0a794b43e39e96a0cbac5d1341/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.478663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd954cc-3224-4252-9a11-877fe9be3319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdebc5336dbb407ebc01513c3e503a94, user_id=28fefb0a794b43e39e96a0cbac5d1341, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.481651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd954cc-3224-4252-9a11-877fe9be3319 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.492190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1101/2192] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:52 2022] PUT /identity/v3/projects/bdebc5336dbb407ebc01513c3e503a94/users/28fefb0a794b43e39e96a0cbac5d1341/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:52.502133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ff022-551a-4d5b-a029-0983ba22a860 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.502420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ff022-551a-4d5b-a029-0983ba22a860 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.502694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ff022-551a-4d5b-a029-0983ba22a860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.528116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:52.535718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9a5ff022-551a-4d5b-a029-0983ba22a860 None None] MFA Rules not processed for user `28fefb0a794b43e39e96a0cbac5d1341`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:52.539019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9a5ff022-551a-4d5b-a029-0983ba22a860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:52.540478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3023e-d87f-47f7-b3c9-2ff0a0ae2a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.543158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3023e-d87f-47f7-b3c9-2ff0a0ae2a0a service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.558673 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1092/2193] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:52.617673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1102/2194] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:52 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:35:52.629859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3023e-d87f-47f7-b3c9-2ff0a0ae2a0a service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.631614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a2f3023e-d87f-47f7-b3c9-2ff0a0ae2a0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:52.642789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3023e-d87f-47f7-b3c9-2ff0a0ae2a0a service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.655019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:52.699121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d93cc-3fcf-4851-bfc7-cb6d716c7bd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.700549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d93cc-3fcf-4851-bfc7-cb6d716c7bd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.700549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d93cc-3fcf-4851-bfc7-cb6d716c7bd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.700549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d93cc-3fcf-4851-bfc7-cb6d716c7bd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.711580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d93cc-3fcf-4851-bfc7-cb6d716c7bd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.714114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d93cc-3fcf-4851-bfc7-cb6d716c7bd9 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.725712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1093/2195] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:52 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:52.902223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ff022-551a-4d5b-a029-0983ba22a860 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:52.912349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ff022-551a-4d5b-a029-0983ba22a860 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:52.915985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6943e-0636-42d1-b0fc-6af905a648b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:52.916660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6943e-0636-42d1-b0fc-6af905a648b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:52.916940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6943e-0636-42d1-b0fc-6af905a648b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:52.917216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6943e-0636-42d1-b0fc-6af905a648b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:52.975523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6943e-0636-42d1-b0fc-6af905a648b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:52.977005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6943e-0636-42d1-b0fc-6af905a648b7 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:52.986209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1103/2196] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:52 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:55.769175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d93cc-3fcf-4851-bfc7-cb6d716c7bd9 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:55.771410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e61d93cc-3fcf-4851-bfc7-cb6d716c7bd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:55.862296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d93cc-3fcf-4851-bfc7-cb6d716c7bd9 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:55.868473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:55.869402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:55.869701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:55.869971 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] PATH_INFO: `/v3/users/9a8aee47a9374451a772cca92c049e7a` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:55.871262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8aee47a9374451a772cca92c049e7a)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:55.873277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:55.923938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8aee47a9374451a772cca92c049e7a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:55.924594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8aee47a9374451a772cca92c049e7a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:55.933895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8aee47a9374451a772cca92c049e7a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:55.934525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8aee47a9374451a772cca92c049e7a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:55.946775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8aee47a9374451a772cca92c049e7a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:55.959787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1094/2197] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:55 2022] DELETE /identity/v3/users/9a8aee47a9374451a772cca92c049e7a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:55.964307 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8eb004-bc9a-41c6-80f2-65cb2de5ca92 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:55.964783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8eb004-bc9a-41c6-80f2-65cb2de5ca92 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:55.964872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8eb004-bc9a-41c6-80f2-65cb2de5ca92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:55.982635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fc8eb004-bc9a-41c6-80f2-65cb2de5ca92 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:55.983979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fc8eb004-bc9a-41c6-80f2-65cb2de5ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:56.046800 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1104/2198] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:56.056591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:56.058286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:56.103198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:56.121555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b62cb-7cc4-4634-9083-9d1fc4eaa6ac admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:56.126028 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00a791-e726-4b90-ac54-e63c7c2cbcb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:56.126671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00a791-e726-4b90-ac54-e63c7c2cbcb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:56.126975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00a791-e726-4b90-ac54-e63c7c2cbcb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:56.127244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00a791-e726-4b90-ac54-e63c7c2cbcb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:56.141174 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00a791-e726-4b90-ac54-e63c7c2cbcb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:56.142634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00a791-e726-4b90-ac54-e63c7c2cbcb1 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:56.153041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1095/2199] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:56.454251 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8eb004-bc9a-41c6-80f2-65cb2de5ca92 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:56.456135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fc8eb004-bc9a-41c6-80f2-65cb2de5ca92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:56.471213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8eb004-bc9a-41c6-80f2-65cb2de5ca92 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:56.476743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:56.477633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:56.477914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:56.478190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] PATH_INFO: `/v3/projects/d4cc6e7692ee40c08e23cc8ed3af4b05` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:56.480121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4cc6e7692ee40c08e23cc8ed3af4b05)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:56.482736 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:56.527050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:56.540373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4cc6e7692ee40c08e23cc8ed3af4b05', 'request_id': 'req-aad7b667-a870-45b3-b080-993335a7b0a3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.546822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4cc6e7692ee40c08e23cc8ed3af4b05', 'request_id': 'req-aad7b667-a870-45b3-b080-993335a7b0a3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.554741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4cc6e7692ee40c08e23cc8ed3af4b05', 'request_id': 'req-aad7b667-a870-45b3-b080-993335a7b0a3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.569449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] The token cache is being invalidate because project d4cc6e7692ee40c08e23cc8ed3af4b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:56.569740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.570386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.573334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1105/2200] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:56 2022] DELETE /identity/v3/projects/d4cc6e7692ee40c08e23cc8ed3af4b05 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:56.577567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00a791-e726-4b90-ac54-e63c7c2cbcb1 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:56.601647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00a791-e726-4b90-ac54-e63c7c2cbcb1 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:56.690339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:56.692193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:56.692193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:56.692193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] PATH_INFO: `/v3/users/9ba269ba5b3a4cf199a91172345821e5` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:56.693738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba269ba5b3a4cf199a91172345821e5)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:56.696067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:56.757839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba269ba5b3a4cf199a91172345821e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.758786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba269ba5b3a4cf199a91172345821e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.767781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba269ba5b3a4cf199a91172345821e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.768474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba269ba5b3a4cf199a91172345821e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.789164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba269ba5b3a4cf199a91172345821e5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:56.813067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1096/2201] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:56 2022] DELETE /identity/v3/users/9ba269ba5b3a4cf199a91172345821e5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:57.187795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:57.189620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:57.225217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad7b667-a870-45b3-b080-993335a7b0a3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:57.302415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:57.303275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:57.303585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:57.303859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] PATH_INFO: `/v3/projects/ca93a8f29f424d239753c64177f75057` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:57.305100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca93a8f29f424d239753c64177f75057)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:57.307018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:57.355817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca93a8f29f424d239753c64177f75057', 'request_id': 'req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.361195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca93a8f29f424d239753c64177f75057', 'request_id': 'req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.367757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca93a8f29f424d239753c64177f75057', 'request_id': 'req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.381586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] The token cache is being invalidate because project ca93a8f29f424d239753c64177f75057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:57.381900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.382548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.384209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1106/2202] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:57 2022] DELETE /identity/v3/projects/ca93a8f29f424d239753c64177f75057 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:57.389280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:57.390735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:57.404323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8951857-b995-474b-8cc0-197b14593076 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:57.471010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:57.472047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:57.472311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:57.472588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] PATH_INFO: `/v3/users/5ce268bf4d8540068490e5c1a7a41175` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:57.473925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce268bf4d8540068490e5c1a7a41175)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:57.475916 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:57.477494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:57.531946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce268bf4d8540068490e5c1a7a41175'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.532592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce268bf4d8540068490e5c1a7a41175'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.540966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce268bf4d8540068490e5c1a7a41175'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.541784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce268bf4d8540068490e5c1a7a41175'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.554768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-421eddb5-a927-4dd9-b9ec-029505501009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce268bf4d8540068490e5c1a7a41175'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:57.567975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1097/2203] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:57 2022] DELETE /identity/v3/users/5ce268bf4d8540068490e5c1a7a41175 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:57.848430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:57.850227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:57.875782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edfd58-d9a4-4778-8723-6aa8d6c040d8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:57.931546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:57.944279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:57.945241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:57.945523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:57.945800 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] PATH_INFO: `/v3/projects/a5ca155944a745b5bc65ab68ece1729e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:57.947007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5ca155944a745b5bc65ab68ece1729e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:57.949008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:57.998846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5ca155944a745b5bc65ab68ece1729e', 'request_id': 'req-368cb564-692c-4a0f-aabc-cb73315748e8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.004866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5ca155944a745b5bc65ab68ece1729e', 'request_id': 'req-368cb564-692c-4a0f-aabc-cb73315748e8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.011375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5ca155944a745b5bc65ab68ece1729e', 'request_id': 'req-368cb564-692c-4a0f-aabc-cb73315748e8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.030293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] The token cache is being invalidate because project a5ca155944a745b5bc65ab68ece1729e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:58.030592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.031245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-368cb564-692c-4a0f-aabc-cb73315748e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.033069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1107/2204] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:57 2022] DELETE /identity/v3/projects/a5ca155944a745b5bc65ab68ece1729e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:35:58.038296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93e0c5-b341-440e-abcf-0e4abb43da12 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.038771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93e0c5-b341-440e-abcf-0e4abb43da12 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.038863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93e0c5-b341-440e-abcf-0e4abb43da12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.056619 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7d93e0c5-b341-440e-abcf-0e4abb43da12 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:58.057800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7d93e0c5-b341-440e-abcf-0e4abb43da12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:58.126641 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1098/2205] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:58.134804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7e8c0-3336-4917-b250-dc490922a784 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.135086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7e8c0-3336-4917-b250-dc490922a784 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.135445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7e8c0-3336-4917-b250-dc490922a784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.154758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-53b7e8c0-3336-4917-b250-dc490922a784 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:58.155898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-53b7e8c0-3336-4917-b250-dc490922a784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:58.165866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1108/2206] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:58.170771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93e0c5-b341-440e-abcf-0e4abb43da12 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:58.181474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93e0c5-b341-440e-abcf-0e4abb43da12 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:58.186539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e674a-62bb-413e-80cc-7ab431250a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:58.187220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e674a-62bb-413e-80cc-7ab431250a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.187523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e674a-62bb-413e-80cc-7ab431250a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.187793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e674a-62bb-413e-80cc-7ab431250a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.188550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e674a-62bb-413e-80cc-7ab431250a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:58.188857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e674a-62bb-413e-80cc-7ab431250a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:58.190453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303e674a-62bb-413e-80cc-7ab431250a4c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:58.203085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1099/2207] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:58.207611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7e8c0-3336-4917-b250-dc490922a784 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:58.217641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7e8c0-3336-4917-b250-dc490922a784 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:58.223396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c09ee-7d54-46cf-9dd5-0098b87b9cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:58.224156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c09ee-7d54-46cf-9dd5-0098b87b9cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.224447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c09ee-7d54-46cf-9dd5-0098b87b9cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.224722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c09ee-7d54-46cf-9dd5-0098b87b9cfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.225390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c09ee-7d54-46cf-9dd5-0098b87b9cfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:58.227341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c09ee-7d54-46cf-9dd5-0098b87b9cfa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:58.256138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1109/2208] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:58 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:58.260990 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e674a-62bb-413e-80cc-7ab431250a4c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:58.271984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e674a-62bb-413e-80cc-7ab431250a4c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:58.332742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5960d-0687-4f49-b205-bab4587b87be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:58.333527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5960d-0687-4f49-b205-bab4587b87be admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.333813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5960d-0687-4f49-b205-bab4587b87be admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.334094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5960d-0687-4f49-b205-bab4587b87be admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.334783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5960d-0687-4f49-b205-bab4587b87be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:58.336742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5960d-0687-4f49-b205-bab4587b87be admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:58.346647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:58.361565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1100/2209] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:58 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:35:58.364914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c09ee-7d54-46cf-9dd5-0098b87b9cfa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:58.375621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c09ee-7d54-46cf-9dd5-0098b87b9cfa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:58.381260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701c175-302b-4928-b952-9210d45b923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:58.381919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701c175-302b-4928-b952-9210d45b923f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.382197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701c175-302b-4928-b952-9210d45b923f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.382467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701c175-302b-4928-b952-9210d45b923f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.383191 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701c175-302b-4928-b952-9210d45b923f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:58.384833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701c175-302b-4928-b952-9210d45b923f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:58.396357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1110/2210] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:58.400591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5960d-0687-4f49-b205-bab4587b87be admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:58.411399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5960d-0687-4f49-b205-bab4587b87be admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:58.417141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19e097-98ac-4751-a2ac-0e114941c60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:58.417872 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19e097-98ac-4751-a2ac-0e114941c60e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.418154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19e097-98ac-4751-a2ac-0e114941c60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.418427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19e097-98ac-4751-a2ac-0e114941c60e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.419162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19e097-98ac-4751-a2ac-0e114941c60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:58.421118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19e097-98ac-4751-a2ac-0e114941c60e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:58.431553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1101/2211] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:58.435751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701c175-302b-4928-b952-9210d45b923f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:58.446611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701c175-302b-4928-b952-9210d45b923f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:58.451910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5380a3-f2d1-46a1-88e0-a4ffa01c5374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:58.452804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5380a3-f2d1-46a1-88e0-a4ffa01c5374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.453077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5380a3-f2d1-46a1-88e0-a4ffa01c5374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.453461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5380a3-f2d1-46a1-88e0-a4ffa01c5374 admin admin] PATH_INFO: `/v3/projects/2085f5f9e71f4eb6b875b584434dd994/users/732246305bce4cc68a327cb71a8c9589/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.469170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5380a3-f2d1-46a1-88e0-a4ffa01c5374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2085f5f9e71f4eb6b875b584434dd994, user_id=732246305bce4cc68a327cb71a8c9589, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:58.472091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5380a3-f2d1-46a1-88e0-a4ffa01c5374 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:58.488916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1111/2212] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:58 2022] PUT /identity/v3/projects/2085f5f9e71f4eb6b875b584434dd994/users/732246305bce4cc68a327cb71a8c9589/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:58.501094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74185673-1e52-40d1-9d6f-37adee85f66e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.501471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74185673-1e52-40d1-9d6f-37adee85f66e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.501865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74185673-1e52-40d1-9d6f-37adee85f66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.533178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-74185673-1e52-40d1-9d6f-37adee85f66e None None] MFA Rules not processed for user `732246305bce4cc68a327cb71a8c9589`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:58.534404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-74185673-1e52-40d1-9d6f-37adee85f66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:58.603929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1102/2213] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:58.618510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5380a3-f2d1-46a1-88e0-a4ffa01c5374 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:58.620440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-be5380a3-f2d1-46a1-88e0-a4ffa01c5374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:58.683055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5380a3-f2d1-46a1-88e0-a4ffa01c5374 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:58.687382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c88ef-42d0-417b-bad0-c10754616f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:58.688035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c88ef-42d0-417b-bad0-c10754616f72 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.688352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c88ef-42d0-417b-bad0-c10754616f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.688585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c88ef-42d0-417b-bad0-c10754616f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.697832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c88ef-42d0-417b-bad0-c10754616f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:58.699265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c88ef-42d0-417b-bad0-c10754616f72 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:58.708620 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1112/2214] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:58.732544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74185673-1e52-40d1-9d6f-37adee85f66e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:58.793025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74185673-1e52-40d1-9d6f-37adee85f66e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:58.801650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:58.803042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.803336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.803662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] PATH_INFO: `/v3/users/732246305bce4cc68a327cb71a8c9589` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.804984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=732246305bce4cc68a327cb71a8c9589)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:58.806986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:58.861608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '732246305bce4cc68a327cb71a8c9589'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.862309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '732246305bce4cc68a327cb71a8c9589'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.871832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '732246305bce4cc68a327cb71a8c9589'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.872435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '732246305bce4cc68a327cb71a8c9589'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.885044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '732246305bce4cc68a327cb71a8c9589'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:58.889745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:58.898674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1103/2215] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:35:58 2022] DELETE /identity/v3/users/732246305bce4cc68a327cb71a8c9589 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:58.903186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15a221-b765-4e08-b33d-5911af0ba6ba None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:58.903506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15a221-b765-4e08-b33d-5911af0ba6ba None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:58.903810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15a221-b765-4e08-b33d-5911af0ba6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:58.921539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4a15a221-b765-4e08-b33d-5911af0ba6ba None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:58.922672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4a15a221-b765-4e08-b33d-5911af0ba6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:58.937263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:58.988987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1113/2216] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:58.999704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:59.001996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:59.062023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a62f39-8cb2-442a-925f-fd6e3714cfe2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:59.066118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8daf2-3e8c-4579-82ea-a98e1b4121c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:59.066774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8daf2-3e8c-4579-82ea-a98e1b4121c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.067057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8daf2-3e8c-4579-82ea-a98e1b4121c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.067424 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8daf2-3e8c-4579-82ea-a98e1b4121c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.077100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8daf2-3e8c-4579-82ea-a98e1b4121c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:59.078598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8daf2-3e8c-4579-82ea-a98e1b4121c6 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:59.088513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1104/2217] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:35:59.363802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15a221-b765-4e08-b33d-5911af0ba6ba None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:59.365327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4a15a221-b765-4e08-b33d-5911af0ba6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:59.377195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15a221-b765-4e08-b33d-5911af0ba6ba None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:59.382108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:59.382978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.383313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.383571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] PATH_INFO: `/v3/projects/2085f5f9e71f4eb6b875b584434dd994` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.384823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2085f5f9e71f4eb6b875b584434dd994)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:59.386748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:59.433491 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2085f5f9e71f4eb6b875b584434dd994', 'request_id': 'req-5208962e-a447-402f-8026-6e95dc5f024b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:59.438775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2085f5f9e71f4eb6b875b584434dd994', 'request_id': 'req-5208962e-a447-402f-8026-6e95dc5f024b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:59.445218 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2085f5f9e71f4eb6b875b584434dd994', 'request_id': 'req-5208962e-a447-402f-8026-6e95dc5f024b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:59.458688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] The token cache is being invalidate because project 2085f5f9e71f4eb6b875b584434dd994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:35:59.458966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:59.459594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5208962e-a447-402f-8026-6e95dc5f024b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:35:59.461156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1114/2218] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:35:59 2022] DELETE /identity/v3/projects/2085f5f9e71f4eb6b875b584434dd994 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:59.466255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948ee05-d8e6-4832-878f-037895904a75 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.466554 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948ee05-d8e6-4832-878f-037895904a75 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.466815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948ee05-d8e6-4832-878f-037895904a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.484450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a948ee05-d8e6-4832-878f-037895904a75 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:59.485682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a948ee05-d8e6-4832-878f-037895904a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:59.551205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1105/2219] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:59.559521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9f142-d7e4-4c80-8840-9160ee6787a3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.559817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9f142-d7e4-4c80-8840-9160ee6787a3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.560093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9f142-d7e4-4c80-8840-9160ee6787a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.580896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-faa9f142-d7e4-4c80-8840-9160ee6787a3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:59.582117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-faa9f142-d7e4-4c80-8840-9160ee6787a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:35:59.592787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1115/2220] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:35:59.597387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948ee05-d8e6-4832-878f-037895904a75 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:59.608898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948ee05-d8e6-4832-878f-037895904a75 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:59.614887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd2d61-b6d6-470b-8d23-716966af2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:59.615639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd2d61-b6d6-470b-8d23-716966af2f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.615924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd2d61-b6d6-470b-8d23-716966af2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.616194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd2d61-b6d6-470b-8d23-716966af2f33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.616935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd2d61-b6d6-470b-8d23-716966af2f33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:35:59.617239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd2d61-b6d6-470b-8d23-716966af2f33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:59.618899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd2d61-b6d6-470b-8d23-716966af2f33 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:59.630587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1106/2221] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:35:59.634950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9f142-d7e4-4c80-8840-9160ee6787a3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:59.644562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9f142-d7e4-4c80-8840-9160ee6787a3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:59.650677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da89a8-acf0-4290-9ebe-728cfdd49b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:59.651425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da89a8-acf0-4290-9ebe-728cfdd49b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.651707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da89a8-acf0-4290-9ebe-728cfdd49b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.651978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da89a8-acf0-4290-9ebe-728cfdd49b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.652673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da89a8-acf0-4290-9ebe-728cfdd49b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:59.654616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da89a8-acf0-4290-9ebe-728cfdd49b45 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:59.689253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1116/2222] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:35:59 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:59.693535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd2d61-b6d6-470b-8d23-716966af2f33 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:59.705533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd2d61-b6d6-470b-8d23-716966af2f33 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:59.765981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b3870-977c-4c11-9cf5-78ffcda68ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:59.766643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b3870-977c-4c11-9cf5-78ffcda68ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.766920 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b3870-977c-4c11-9cf5-78ffcda68ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.767187 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b3870-977c-4c11-9cf5-78ffcda68ba4 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.767933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b3870-977c-4c11-9cf5-78ffcda68ba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:59.769829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b3870-977c-4c11-9cf5-78ffcda68ba4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:59.793875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1107/2223] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:35:59 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:35:59.798767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da89a8-acf0-4290-9ebe-728cfdd49b45 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:59.809753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da89a8-acf0-4290-9ebe-728cfdd49b45 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:59.815412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53929a-1479-4182-a482-05827e0aa096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:59.816073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53929a-1479-4182-a482-05827e0aa096 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.816354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53929a-1479-4182-a482-05827e0aa096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.816625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53929a-1479-4182-a482-05827e0aa096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.817678 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53929a-1479-4182-a482-05827e0aa096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:59.819019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53929a-1479-4182-a482-05827e0aa096 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:59.829969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1117/2224] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:59.834404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b3870-977c-4c11-9cf5-78ffcda68ba4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:59.849192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b3870-977c-4c11-9cf5-78ffcda68ba4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:59.853915 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639a1c9-fb62-4635-a9a0-5575178673ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:59.854563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639a1c9-fb62-4635-a9a0-5575178673ad admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.854828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639a1c9-fb62-4635-a9a0-5575178673ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.855123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639a1c9-fb62-4635-a9a0-5575178673ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.855866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639a1c9-fb62-4635-a9a0-5575178673ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:59.857447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639a1c9-fb62-4635-a9a0-5575178673ad admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:59.868067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1108/2225] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:35:59.872226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53929a-1479-4182-a482-05827e0aa096 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:35:59.891170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53929a-1479-4182-a482-05827e0aa096 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:35:59.897047 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c52930-9fa1-426a-af39-9c7a24f3a4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:35:59.897939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c52930-9fa1-426a-af39-9c7a24f3a4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.898217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c52930-9fa1-426a-af39-9c7a24f3a4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.898487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c52930-9fa1-426a-af39-9c7a24f3a4f7 admin admin] PATH_INFO: `/v3/projects/54afb43716fa487bb7b8e3cfcea65a01/users/01a61d25cce84864856f26a7dee55a1e/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.914851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c52930-9fa1-426a-af39-9c7a24f3a4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54afb43716fa487bb7b8e3cfcea65a01, user_id=01a61d25cce84864856f26a7dee55a1e, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:35:59.917582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c52930-9fa1-426a-af39-9c7a24f3a4f7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:35:59.931703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1118/2226] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:35:59 2022] PUT /identity/v3/projects/54afb43716fa487bb7b8e3cfcea65a01/users/01a61d25cce84864856f26a7dee55a1e/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:35:59.943553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4c4dd-3059-4d57-88e2-6bba8374287e None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:35:59.943874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4c4dd-3059-4d57-88e2-6bba8374287e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:35:59.944222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4c4dd-3059-4d57-88e2-6bba8374287e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:35:59.950999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:35:59.973523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-73d4c4dd-3059-4d57-88e2-6bba8374287e None None] MFA Rules not processed for user `01a61d25cce84864856f26a7dee55a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:35:59.974875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-73d4c4dd-3059-4d57-88e2-6bba8374287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:00.046654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1109/2227] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:35:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:00.057917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c52930-9fa1-426a-af39-9c7a24f3a4f7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:00.059784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b4c52930-9fa1-426a-af39-9c7a24f3a4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:00.102111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:00.118878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c52930-9fa1-426a-af39-9c7a24f3a4f7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:00.122684 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a92b1-9cc0-44ca-9e21-8c93a26c471e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:00.123339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a92b1-9cc0-44ca-9e21-8c93a26c471e service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:00.123640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a92b1-9cc0-44ca-9e21-8c93a26c471e service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:00.123915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a92b1-9cc0-44ca-9e21-8c93a26c471e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:00.132993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a92b1-9cc0-44ca-9e21-8c93a26c471e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:00.134462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a92b1-9cc0-44ca-9e21-8c93a26c471e service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:00.144018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1119/2228] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:10.038288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=31 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:10.139604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=8 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:11.871949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4c4dd-3059-4d57-88e2-6bba8374287e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:11.873451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-73d4c4dd-3059-4d57-88e2-6bba8374287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:11.937793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4c4dd-3059-4d57-88e2-6bba8374287e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:11.942416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:11.943310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:11.943590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:11.943919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] PATH_INFO: `/v3/users/184e4c13c82e4c57aed348b882284a51` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:11.945323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=184e4c13c82e4c57aed348b882284a51)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:11.947014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:11.995551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184e4c13c82e4c57aed348b882284a51'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:11.996266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184e4c13c82e4c57aed348b882284a51'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.005658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184e4c13c82e4c57aed348b882284a51'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.006616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184e4c13c82e4c57aed348b882284a51'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.021529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184e4c13c82e4c57aed348b882284a51'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.035116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1110/2229] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:11 2022] DELETE /identity/v3/users/184e4c13c82e4c57aed348b882284a51 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:12.040084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb41bb-a4ba-45e4-b44d-7ae80f819bec None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:12.040575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb41bb-a4ba-45e4-b44d-7ae80f819bec None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:12.040738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb41bb-a4ba-45e4-b44d-7ae80f819bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:12.059928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-17bb41bb-a4ba-45e4-b44d-7ae80f819bec None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:12.071913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-17bb41bb-a4ba-45e4-b44d-7ae80f819bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:12.147256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1120/2230] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:12.156664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:12.158094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:12.216743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542a4a6-3a9a-40a9-aff9-b0f37a4c1b29 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:12.220862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff0a28-b199-4793-ba0b-e1169fe2c55c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:12.221493 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff0a28-b199-4793-ba0b-e1169fe2c55c service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:12.221789 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff0a28-b199-4793-ba0b-e1169fe2c55c service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:12.222063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff0a28-b199-4793-ba0b-e1169fe2c55c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:12.234065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff0a28-b199-4793-ba0b-e1169fe2c55c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:12.235822 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff0a28-b199-4793-ba0b-e1169fe2c55c service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:12.246547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1111/2231] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:12.531913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb41bb-a4ba-45e4-b44d-7ae80f819bec None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:12.533408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-17bb41bb-a4ba-45e4-b44d-7ae80f819bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:12.546565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb41bb-a4ba-45e4-b44d-7ae80f819bec None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:12.552490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:12.553556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:12.553841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:12.554657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] PATH_INFO: `/v3/projects/b0facc0864d7407cbdfac93d17cdee82` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:12.555738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0facc0864d7407cbdfac93d17cdee82)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:12.557805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:12.608002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0facc0864d7407cbdfac93d17cdee82', 'request_id': 'req-93214172-53a1-452a-8337-ac5b3edd6c27'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.614037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0facc0864d7407cbdfac93d17cdee82', 'request_id': 'req-93214172-53a1-452a-8337-ac5b3edd6c27'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.620602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0facc0864d7407cbdfac93d17cdee82', 'request_id': 'req-93214172-53a1-452a-8337-ac5b3edd6c27'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.634609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] The token cache is being invalidate because project b0facc0864d7407cbdfac93d17cdee82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:12.634898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.635575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.637149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1121/2232] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:12 2022] DELETE /identity/v3/projects/b0facc0864d7407cbdfac93d17cdee82 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:12.641159 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff0a28-b199-4793-ba0b-e1169fe2c55c service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:12.651011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff0a28-b199-4793-ba0b-e1169fe2c55c service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:12.670056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:12.711458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:12.712343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:12.712621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:12.712894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] PATH_INFO: `/v3/users/28fefb0a794b43e39e96a0cbac5d1341` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:12.714148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28fefb0a794b43e39e96a0cbac5d1341)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:12.716826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:12.777474 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28fefb0a794b43e39e96a0cbac5d1341'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.778170 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28fefb0a794b43e39e96a0cbac5d1341'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.785071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28fefb0a794b43e39e96a0cbac5d1341'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.785722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28fefb0a794b43e39e96a0cbac5d1341'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.798885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2799f3b9-258b-4916-8815-ecb129a2f2f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28fefb0a794b43e39e96a0cbac5d1341'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:12.820317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1112/2233] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:12 2022] DELETE /identity/v3/users/28fefb0a794b43e39e96a0cbac5d1341 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:13.154100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:13.156345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:13.183281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93214172-53a1-452a-8337-ac5b3edd6c27 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:13.244460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:13.245334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.245615 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.245925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] PATH_INFO: `/v3/projects/bdebc5336dbb407ebc01513c3e503a94` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.247107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdebc5336dbb407ebc01513c3e503a94)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:13.249104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:13.284247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:13.304844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdebc5336dbb407ebc01513c3e503a94', 'request_id': 'req-440f4ab3-51a4-499a-9a23-4ded5879c61e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:13.311169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdebc5336dbb407ebc01513c3e503a94', 'request_id': 'req-440f4ab3-51a4-499a-9a23-4ded5879c61e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:13.317392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdebc5336dbb407ebc01513c3e503a94', 'request_id': 'req-440f4ab3-51a4-499a-9a23-4ded5879c61e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:13.330009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] The token cache is being invalidate because project bdebc5336dbb407ebc01513c3e503a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:13.330295 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:13.330904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-440f4ab3-51a4-499a-9a23-4ded5879c61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:13.332528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1122/2234] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:13 2022] DELETE /identity/v3/projects/bdebc5336dbb407ebc01513c3e503a94 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:13.337551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326f83b-6970-4983-b86a-b959674b333f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.338050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326f83b-6970-4983-b86a-b959674b333f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.338234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326f83b-6970-4983-b86a-b959674b333f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.356611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6326f83b-6970-4983-b86a-b959674b333f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:13.357809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6326f83b-6970-4983-b86a-b959674b333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:13.421927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1113/2235] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:13.430128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca63131-59a2-4441-b08b-d1710b4868e1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.430419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca63131-59a2-4441-b08b-d1710b4868e1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.430720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca63131-59a2-4441-b08b-d1710b4868e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.450093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8ca63131-59a2-4441-b08b-d1710b4868e1 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:13.451316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8ca63131-59a2-4441-b08b-d1710b4868e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:13.459821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1123/2236] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:13.464119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326f83b-6970-4983-b86a-b959674b333f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:13.475331 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326f83b-6970-4983-b86a-b959674b333f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:13.480654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668032d9-8930-48ed-84cd-629d84038d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:13.481320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668032d9-8930-48ed-84cd-629d84038d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.481607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668032d9-8930-48ed-84cd-629d84038d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.481884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668032d9-8930-48ed-84cd-629d84038d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.482616 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668032d9-8930-48ed-84cd-629d84038d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:36:13.482930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668032d9-8930-48ed-84cd-629d84038d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:13.484613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668032d9-8930-48ed-84cd-629d84038d9c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:13.497425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1114/2237] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:36:13.504093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca63131-59a2-4441-b08b-d1710b4868e1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:13.519258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca63131-59a2-4441-b08b-d1710b4868e1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:13.526500 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a021a-1689-4265-b2ac-6fb373cba8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:13.527171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a021a-1689-4265-b2ac-6fb373cba8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.527487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a021a-1689-4265-b2ac-6fb373cba8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.527751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a021a-1689-4265-b2ac-6fb373cba8f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.528458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a021a-1689-4265-b2ac-6fb373cba8f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:13.530426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a021a-1689-4265-b2ac-6fb373cba8f1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:13.560988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1124/2238] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:13 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:13.565084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668032d9-8930-48ed-84cd-629d84038d9c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:13.575663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668032d9-8930-48ed-84cd-629d84038d9c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:13.601084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:13.644314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef4093-974a-4105-b1b8-1db36dafcf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:13.645183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef4093-974a-4105-b1b8-1db36dafcf44 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.645525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef4093-974a-4105-b1b8-1db36dafcf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.645885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef4093-974a-4105-b1b8-1db36dafcf44 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.646858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef4093-974a-4105-b1b8-1db36dafcf44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:13.649241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef4093-974a-4105-b1b8-1db36dafcf44 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:13.671731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1115/2239] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:13 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:13.676173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a021a-1689-4265-b2ac-6fb373cba8f1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:13.687007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a021a-1689-4265-b2ac-6fb373cba8f1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:13.691885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6943-5f3a-4416-888a-0dd0d6220c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:13.692538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6943-5f3a-4416-888a-0dd0d6220c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.692822 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6943-5f3a-4416-888a-0dd0d6220c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.693086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6943-5f3a-4416-888a-0dd0d6220c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.693862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae6943-5f3a-4416-888a-0dd0d6220c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:13.695495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae6943-5f3a-4416-888a-0dd0d6220c95 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:13.707468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1125/2240] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:13.712261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef4093-974a-4105-b1b8-1db36dafcf44 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:13.722790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef4093-974a-4105-b1b8-1db36dafcf44 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:13.728390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6dc96-3501-4b48-b2d1-d079b0364e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:13.729055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6dc96-3501-4b48-b2d1-d079b0364e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.729341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6dc96-3501-4b48-b2d1-d079b0364e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.729612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6dc96-3501-4b48-b2d1-d079b0364e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.730354 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6dc96-3501-4b48-b2d1-d079b0364e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:13.732035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6dc96-3501-4b48-b2d1-d079b0364e5d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:13.742878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1116/2241] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:36:13.747123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6943-5f3a-4416-888a-0dd0d6220c95 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:13.757948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6943-5f3a-4416-888a-0dd0d6220c95 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:13.763867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc648609-9417-40bf-9f51-09d6e85291c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:13.764753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc648609-9417-40bf-9f51-09d6e85291c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.765034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc648609-9417-40bf-9f51-09d6e85291c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.765305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc648609-9417-40bf-9f51-09d6e85291c9 admin admin] PATH_INFO: `/v3/projects/e7196b49b5764ab3b33484f5cf311747/users/e2be7ad1d0a84ca996df5c80eae7506a/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.780391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc648609-9417-40bf-9f51-09d6e85291c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7196b49b5764ab3b33484f5cf311747, user_id=e2be7ad1d0a84ca996df5c80eae7506a, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:13.783080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc648609-9417-40bf-9f51-09d6e85291c9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:13.799601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1126/2242] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:13 2022] PUT /identity/v3/projects/e7196b49b5764ab3b33484f5cf311747/users/e2be7ad1d0a84ca996df5c80eae7506a/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:13.809994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498a7b0-b337-4d7c-828f-e71df7a16114 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.810283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498a7b0-b337-4d7c-828f-e71df7a16114 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.810539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498a7b0-b337-4d7c-828f-e71df7a16114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.837754 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7498a7b0-b337-4d7c-828f-e71df7a16114 None None] MFA Rules not processed for user `e2be7ad1d0a84ca996df5c80eae7506a`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:13.838901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7498a7b0-b337-4d7c-828f-e71df7a16114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:13.903024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1117/2243] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:13.907464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc648609-9417-40bf-9f51-09d6e85291c9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:13.977081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc648609-9417-40bf-9f51-09d6e85291c9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:13.984261 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b9873-8ec2-44c1-b4fe-a71de2cef0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:13.984935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b9873-8ec2-44c1-b4fe-a71de2cef0cb admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:13.985219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b9873-8ec2-44c1-b4fe-a71de2cef0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:13.985496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b9873-8ec2-44c1-b4fe-a71de2cef0cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:13.986191 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b9873-8ec2-44c1-b4fe-a71de2cef0cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:13.988146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b9873-8ec2-44c1-b4fe-a71de2cef0cb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:14.018589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1127/2244] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:13 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:14.022633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498a7b0-b337-4d7c-828f-e71df7a16114 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:14.033225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498a7b0-b337-4d7c-828f-e71df7a16114 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:14.092278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cda71-4572-458b-a016-7ee7fccc0c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:14.092855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cda71-4572-458b-a016-7ee7fccc0c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:14.093131 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cda71-4572-458b-a016-7ee7fccc0c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:14.093403 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77cda71-4572-458b-a016-7ee7fccc0c46 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:14.094082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cda71-4572-458b-a016-7ee7fccc0c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:14.096073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77cda71-4572-458b-a016-7ee7fccc0c46 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:14.121994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1118/2245] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:14 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:14.126199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b9873-8ec2-44c1-b4fe-a71de2cef0cb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:14.137171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b9873-8ec2-44c1-b4fe-a71de2cef0cb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:14.142517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccb4fa-3f93-4bef-8e1c-011d5fe40b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:14.143200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccb4fa-3f93-4bef-8e1c-011d5fe40b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:14.143513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccb4fa-3f93-4bef-8e1c-011d5fe40b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:14.143795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccb4fa-3f93-4bef-8e1c-011d5fe40b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:14.144545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccb4fa-3f93-4bef-8e1c-011d5fe40b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:14.146165 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebccb4fa-3f93-4bef-8e1c-011d5fe40b37 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:14.157764 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1128/2246] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:14.162261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cda71-4572-458b-a016-7ee7fccc0c46 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:14.172821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77cda71-4572-458b-a016-7ee7fccc0c46 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:14.178017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7788da-af39-427e-be35-f3dfb8457848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:14.178957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7788da-af39-427e-be35-f3dfb8457848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:14.179234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7788da-af39-427e-be35-f3dfb8457848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:14.179544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7788da-af39-427e-be35-f3dfb8457848 admin admin] PATH_INFO: `/v3/projects/31efd41af2344fd5b3d5646f0c5d0126/users/fc5883e3dedc4e48a34f9fc5a4568ef5/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:14.194352 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7788da-af39-427e-be35-f3dfb8457848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31efd41af2344fd5b3d5646f0c5d0126, user_id=fc5883e3dedc4e48a34f9fc5a4568ef5, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:14.197087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7788da-af39-427e-be35-f3dfb8457848 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:14.211621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1119/2247] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:14 2022] PUT /identity/v3/projects/31efd41af2344fd5b3d5646f0c5d0126/users/fc5883e3dedc4e48a34f9fc5a4568ef5/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:14.215856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccb4fa-3f93-4bef-8e1c-011d5fe40b37 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:14.226908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccb4fa-3f93-4bef-8e1c-011d5fe40b37 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:14.247583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:14.285145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4254a3-f87c-4518-9a32-3f329815c88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:14.285818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4254a3-f87c-4518-9a32-3f329815c88d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:14.286069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4254a3-f87c-4518-9a32-3f329815c88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:14.286342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4254a3-f87c-4518-9a32-3f329815c88d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:14.287191 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4254a3-f87c-4518-9a32-3f329815c88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:14.289008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4254a3-f87c-4518-9a32-3f329815c88d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:14.299760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1129/2248] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:14.304219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7788da-af39-427e-be35-f3dfb8457848 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:14.310084 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:14.316888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7788da-af39-427e-be35-f3dfb8457848 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:14.322358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9a275-f873-4fb5-9c2a-aa48804e98a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:14.323237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9a275-f873-4fb5-9c2a-aa48804e98a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:14.323553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9a275-f873-4fb5-9c2a-aa48804e98a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:14.323892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9a275-f873-4fb5-9c2a-aa48804e98a0 admin admin] PATH_INFO: `/v3/projects/31efd41af2344fd5b3d5646f0c5d0126/users/fc5883e3dedc4e48a34f9fc5a4568ef5/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:14.325947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9a275-f873-4fb5-9c2a-aa48804e98a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31efd41af2344fd5b3d5646f0c5d0126, user_id=fc5883e3dedc4e48a34f9fc5a4568ef5, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:14.328626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9a275-f873-4fb5-9c2a-aa48804e98a0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:14.339286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1120/2249] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:14 2022] PUT /identity/v3/projects/31efd41af2344fd5b3d5646f0c5d0126/users/fc5883e3dedc4e48a34f9fc5a4568ef5/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:14.349662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a5017-8132-4a68-bd5f-2e7870cf6fd8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:14.349886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a5017-8132-4a68-bd5f-2e7870cf6fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:14.350177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a5017-8132-4a68-bd5f-2e7870cf6fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:14.377874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-099a5017-8132-4a68-bd5f-2e7870cf6fd8 None None] MFA Rules not processed for user `fc5883e3dedc4e48a34f9fc5a4568ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:14.379075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-099a5017-8132-4a68-bd5f-2e7870cf6fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:14.463801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1130/2250] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:14.475006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9a275-f873-4fb5-9c2a-aa48804e98a0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:14.477280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0be9a275-f873-4fb5-9c2a-aa48804e98a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:14.534697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9a275-f873-4fb5-9c2a-aa48804e98a0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:14.538709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff9a76-b17f-41b6-9aa1-286a75ae85b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:14.539266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff9a76-b17f-41b6-9aa1-286a75ae85b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:14.539583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff9a76-b17f-41b6-9aa1-286a75ae85b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:14.539865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff9a76-b17f-41b6-9aa1-286a75ae85b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:14.550067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff9a76-b17f-41b6-9aa1-286a75ae85b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:14.551583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff9a76-b17f-41b6-9aa1-286a75ae85b4 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:14.561811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1121/2251] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:14.650650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a5017-8132-4a68-bd5f-2e7870cf6fd8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:14.661230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a5017-8132-4a68-bd5f-2e7870cf6fd8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:14.665901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc19d89-1326-494b-a03e-6bb6c9aa7c57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:14.666591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc19d89-1326-494b-a03e-6bb6c9aa7c57 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:14.666924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc19d89-1326-494b-a03e-6bb6c9aa7c57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:14.667176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc19d89-1326-494b-a03e-6bb6c9aa7c57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:14.727247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc19d89-1326-494b-a03e-6bb6c9aa7c57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:14.728751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc19d89-1326-494b-a03e-6bb6c9aa7c57 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:14.738030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1131/2252] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:18.206617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff9a76-b17f-41b6-9aa1-286a75ae85b4 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:18.211522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9cff9a76-b17f-41b6-9aa1-286a75ae85b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:18.282719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff9a76-b17f-41b6-9aa1-286a75ae85b4 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:18.287408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:18.288279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:18.288574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:18.288845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] PATH_INFO: `/v3/users/abeeca8832e54b7ebf1aebc6d1f0e9af` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:18.290141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abeeca8832e54b7ebf1aebc6d1f0e9af)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:18.292083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:18.348283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abeeca8832e54b7ebf1aebc6d1f0e9af'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:18.349494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abeeca8832e54b7ebf1aebc6d1f0e9af'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:18.358880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abeeca8832e54b7ebf1aebc6d1f0e9af'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:18.359640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abeeca8832e54b7ebf1aebc6d1f0e9af'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:18.374307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abeeca8832e54b7ebf1aebc6d1f0e9af'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:18.388505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1122/2253] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:18 2022] DELETE /identity/v3/users/abeeca8832e54b7ebf1aebc6d1f0e9af => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:18.393230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad73d72-4239-4586-9c27-ee10ee308d8e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:18.393740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad73d72-4239-4586-9c27-ee10ee308d8e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:18.393855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad73d72-4239-4586-9c27-ee10ee308d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:18.420250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3ad73d72-4239-4586-9c27-ee10ee308d8e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:18.421505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3ad73d72-4239-4586-9c27-ee10ee308d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:18.462391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:18.484150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1132/2254] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:18.494711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:18.496096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:18.510821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:18.565421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334491c-f10b-4901-bf73-17ee6ebdd290 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:18.569632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d35464f-2b0e-4b8d-9f94-390cccfe87e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:18.570410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35464f-2b0e-4b8d-9f94-390cccfe87e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:18.570584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35464f-2b0e-4b8d-9f94-390cccfe87e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:18.570803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d35464f-2b0e-4b8d-9f94-390cccfe87e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:18.582130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35464f-2b0e-4b8d-9f94-390cccfe87e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:18.583605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d35464f-2b0e-4b8d-9f94-390cccfe87e0 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:18.593954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1123/2255] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:36:18.886227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad73d72-4239-4586-9c27-ee10ee308d8e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:18.887900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3ad73d72-4239-4586-9c27-ee10ee308d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:18.904320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad73d72-4239-4586-9c27-ee10ee308d8e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:18.909852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:18.910741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:18.911019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:18.911323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] PATH_INFO: `/v3/projects/87f3bfb730144616b5e329cf0cfd37ca` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:18.912645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f3bfb730144616b5e329cf0cfd37ca)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:18.914712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:18.971834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f3bfb730144616b5e329cf0cfd37ca', 'request_id': 'req-63758298-0c3b-4cb4-8642-70af228765de'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:18.977753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f3bfb730144616b5e329cf0cfd37ca', 'request_id': 'req-63758298-0c3b-4cb4-8642-70af228765de'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:18.984146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f3bfb730144616b5e329cf0cfd37ca', 'request_id': 'req-63758298-0c3b-4cb4-8642-70af228765de'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:18.999730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] The token cache is being invalidate because project 87f3bfb730144616b5e329cf0cfd37ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:19.000014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:19.000682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63758298-0c3b-4cb4-8642-70af228765de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:19.002188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1133/2256] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:18 2022] DELETE /identity/v3/projects/87f3bfb730144616b5e329cf0cfd37ca => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:19.007873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0f2c-11f6-4a5a-82a6-5cc82d26d1bb None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.007873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0f2c-11f6-4a5a-82a6-5cc82d26d1bb None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.008185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d0f2c-11f6-4a5a-82a6-5cc82d26d1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.030653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-186d0f2c-11f6-4a5a-82a6-5cc82d26d1bb None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:19.031888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-186d0f2c-11f6-4a5a-82a6-5cc82d26d1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:19.098125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1124/2257] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:19.107274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043b172-00b4-4cfe-a63a-afa56c1d907a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.107607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043b172-00b4-4cfe-a63a-afa56c1d907a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.107875 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043b172-00b4-4cfe-a63a-afa56c1d907a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.127604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9043b172-00b4-4cfe-a63a-afa56c1d907a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:19.128814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9043b172-00b4-4cfe-a63a-afa56c1d907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:19.137274 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1134/2258] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:19.141702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0f2c-11f6-4a5a-82a6-5cc82d26d1bb None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:19.151635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d0f2c-11f6-4a5a-82a6-5cc82d26d1bb None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:19.156584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165816d-64c2-48cc-b575-422015b47735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:19.157235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165816d-64c2-48cc-b575-422015b47735 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.157517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165816d-64c2-48cc-b575-422015b47735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.157788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165816d-64c2-48cc-b575-422015b47735 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.158507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165816d-64c2-48cc-b575-422015b47735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:36:19.158814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165816d-64c2-48cc-b575-422015b47735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:19.160549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165816d-64c2-48cc-b575-422015b47735 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:19.173152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1125/2259] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:36:19.177680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043b172-00b4-4cfe-a63a-afa56c1d907a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:19.188116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043b172-00b4-4cfe-a63a-afa56c1d907a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:19.192920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a12942-4e2c-44a1-93a1-8ceb59c8600d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:19.193567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a12942-4e2c-44a1-93a1-8ceb59c8600d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.193852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a12942-4e2c-44a1-93a1-8ceb59c8600d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.194126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a12942-4e2c-44a1-93a1-8ceb59c8600d admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.194841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a12942-4e2c-44a1-93a1-8ceb59c8600d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:19.196914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a12942-4e2c-44a1-93a1-8ceb59c8600d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:19.224398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1135/2260] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:19 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:19.228724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165816d-64c2-48cc-b575-422015b47735 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:19.240499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165816d-64c2-48cc-b575-422015b47735 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:19.296730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ad38-db41-4031-a2a6-ec9e9235afeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:19.297418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ad38-db41-4031-a2a6-ec9e9235afeb admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.297700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ad38-db41-4031-a2a6-ec9e9235afeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.297968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ad38-db41-4031-a2a6-ec9e9235afeb admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.298644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98ad38-db41-4031-a2a6-ec9e9235afeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:19.300605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98ad38-db41-4031-a2a6-ec9e9235afeb admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:19.325249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1126/2261] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:19 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:36:19.327683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a12942-4e2c-44a1-93a1-8ceb59c8600d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:19.338368 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a12942-4e2c-44a1-93a1-8ceb59c8600d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:19.343475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e7571-924c-4bd1-8eef-77a7d9f9a839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:19.344125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e7571-924c-4bd1-8eef-77a7d9f9a839 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.344401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e7571-924c-4bd1-8eef-77a7d9f9a839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.344671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e7571-924c-4bd1-8eef-77a7d9f9a839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.345398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e7571-924c-4bd1-8eef-77a7d9f9a839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:19.346992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e7571-924c-4bd1-8eef-77a7d9f9a839 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:19.357639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1136/2262] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:19.361594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ad38-db41-4031-a2a6-ec9e9235afeb admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:19.371680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ad38-db41-4031-a2a6-ec9e9235afeb admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:19.377009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2732b-53ce-4ff2-8a18-a91a9511452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:19.377658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2732b-53ce-4ff2-8a18-a91a9511452c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.377943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2732b-53ce-4ff2-8a18-a91a9511452c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.378217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2732b-53ce-4ff2-8a18-a91a9511452c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.378933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2732b-53ce-4ff2-8a18-a91a9511452c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:19.380665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2732b-53ce-4ff2-8a18-a91a9511452c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:19.392476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1127/2263] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:36:19.397471 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e7571-924c-4bd1-8eef-77a7d9f9a839 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:19.415222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e7571-924c-4bd1-8eef-77a7d9f9a839 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:19.421392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1193678-d55c-4085-b955-5ae33f1449ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:19.422275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193678-d55c-4085-b955-5ae33f1449ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.422560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193678-d55c-4085-b955-5ae33f1449ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.422838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193678-d55c-4085-b955-5ae33f1449ed admin admin] PATH_INFO: `/v3/projects/b791a494c7c442269d567f72e37f03ca/users/3ac45addb60a407f92321a5da9b47144/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.440806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1193678-d55c-4085-b955-5ae33f1449ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b791a494c7c442269d567f72e37f03ca, user_id=3ac45addb60a407f92321a5da9b47144, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:19.443514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1193678-d55c-4085-b955-5ae33f1449ed admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:19.459928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1137/2264] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:19 2022] PUT /identity/v3/projects/b791a494c7c442269d567f72e37f03ca/users/3ac45addb60a407f92321a5da9b47144/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:19.471544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c254f4a-e045-44e6-a695-d5ed2be0fd10 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.471544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c254f4a-e045-44e6-a695-d5ed2be0fd10 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.471845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c254f4a-e045-44e6-a695-d5ed2be0fd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.504933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4c254f4a-e045-44e6-a695-d5ed2be0fd10 None None] MFA Rules not processed for user `3ac45addb60a407f92321a5da9b47144`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:19.506167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4c254f4a-e045-44e6-a695-d5ed2be0fd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:19.522910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:19.588932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1128/2265] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:19.600544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1193678-d55c-4085-b955-5ae33f1449ed admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:19.602383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b1193678-d55c-4085-b955-5ae33f1449ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:19.682190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:19.690435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1193678-d55c-4085-b955-5ae33f1449ed admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:19.697349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cb872-b076-4213-8a1c-da257c663671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:19.698000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cb872-b076-4213-8a1c-da257c663671 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:19.698277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cb872-b076-4213-8a1c-da257c663671 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:19.698549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cb872-b076-4213-8a1c-da257c663671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:19.706728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021cb872-b076-4213-8a1c-da257c663671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:19.708348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021cb872-b076-4213-8a1c-da257c663671 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:19.717638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1138/2266] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:19.946874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c254f4a-e045-44e6-a695-d5ed2be0fd10 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:20.017200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c254f4a-e045-44e6-a695-d5ed2be0fd10 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:20.021795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:20.022667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:20.022948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:20.023225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] PATH_INFO: `/v3/users/3ac45addb60a407f92321a5da9b47144` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:20.024998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac45addb60a407f92321a5da9b47144)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:20.027258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:20.075441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac45addb60a407f92321a5da9b47144'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.076226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac45addb60a407f92321a5da9b47144'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.084601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac45addb60a407f92321a5da9b47144'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.085238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac45addb60a407f92321a5da9b47144'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.097413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac45addb60a407f92321a5da9b47144'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.115734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1129/2267] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:19 2022] DELETE /identity/v3/users/3ac45addb60a407f92321a5da9b47144 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:20.120498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff85424-3061-4aa3-af5c-45bb353a2b3e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:20.120913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff85424-3061-4aa3-af5c-45bb353a2b3e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:20.121314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff85424-3061-4aa3-af5c-45bb353a2b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:20.144706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cff85424-3061-4aa3-af5c-45bb353a2b3e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:20.145915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cff85424-3061-4aa3-af5c-45bb353a2b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:20.207469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1139/2268] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:20.217323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:20.218923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:20.263356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:20.292247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b53ee0-cd8c-49fa-9657-524cfe5e99d7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:20.296458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668f74f-a5a7-4cae-9b03-d1cd92ea2956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:20.297122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668f74f-a5a7-4cae-9b03-d1cd92ea2956 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:20.297428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668f74f-a5a7-4cae-9b03-d1cd92ea2956 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:20.297700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668f74f-a5a7-4cae-9b03-d1cd92ea2956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:20.307584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668f74f-a5a7-4cae-9b03-d1cd92ea2956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:20.309034 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668f74f-a5a7-4cae-9b03-d1cd92ea2956 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:20.318867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1130/2269] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:20.701691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff85424-3061-4aa3-af5c-45bb353a2b3e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:20.703222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cff85424-3061-4aa3-af5c-45bb353a2b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:20.715470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff85424-3061-4aa3-af5c-45bb353a2b3e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:20.721287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:20.722162 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:20.722452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:20.722704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] PATH_INFO: `/v3/projects/b791a494c7c442269d567f72e37f03ca` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:20.724127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b791a494c7c442269d567f72e37f03ca)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:20.725951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:20.775625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b791a494c7c442269d567f72e37f03ca', 'request_id': 'req-846ee46a-c2ed-4037-9ada-b2c126d16c78'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.780975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b791a494c7c442269d567f72e37f03ca', 'request_id': 'req-846ee46a-c2ed-4037-9ada-b2c126d16c78'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.787050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b791a494c7c442269d567f72e37f03ca', 'request_id': 'req-846ee46a-c2ed-4037-9ada-b2c126d16c78'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.800947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] The token cache is being invalidate because project b791a494c7c442269d567f72e37f03ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:20.801254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.801866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-846ee46a-c2ed-4037-9ada-b2c126d16c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:20.803472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1140/2270] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:20 2022] DELETE /identity/v3/projects/b791a494c7c442269d567f72e37f03ca => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:20.808526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347aceaa-964d-443c-b430-517b69a3ff17 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:20.808822 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347aceaa-964d-443c-b430-517b69a3ff17 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:20.809105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347aceaa-964d-443c-b430-517b69a3ff17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:20.826729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-347aceaa-964d-443c-b430-517b69a3ff17 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:20.827973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-347aceaa-964d-443c-b430-517b69a3ff17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:20.887799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1131/2271] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:20.895673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f741d-1ad7-41ed-af34-9be4bb7fad77 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:20.895962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f741d-1ad7-41ed-af34-9be4bb7fad77 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:20.896243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f741d-1ad7-41ed-af34-9be4bb7fad77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:20.914920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-992f741d-1ad7-41ed-af34-9be4bb7fad77 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:20.916167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-992f741d-1ad7-41ed-af34-9be4bb7fad77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:20.924954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1141/2272] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:20.929529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347aceaa-964d-443c-b430-517b69a3ff17 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:20.940083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347aceaa-964d-443c-b430-517b69a3ff17 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:20.948042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16223a69-01b1-41e2-a0d2-9346f45713e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:20.948762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16223a69-01b1-41e2-a0d2-9346f45713e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:20.949163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16223a69-01b1-41e2-a0d2-9346f45713e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:20.949560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16223a69-01b1-41e2-a0d2-9346f45713e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:20.950339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16223a69-01b1-41e2-a0d2-9346f45713e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:36:20.950657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16223a69-01b1-41e2-a0d2-9346f45713e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:20.952341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16223a69-01b1-41e2-a0d2-9346f45713e4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:20.967153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1132/2273] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:36:20.971582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f741d-1ad7-41ed-af34-9be4bb7fad77 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:20.981993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f741d-1ad7-41ed-af34-9be4bb7fad77 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:20.987653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18cd67-515d-4f01-bafe-1dbf2dc6b056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:20.988355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18cd67-515d-4f01-bafe-1dbf2dc6b056 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:20.988646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18cd67-515d-4f01-bafe-1dbf2dc6b056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:20.988923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18cd67-515d-4f01-bafe-1dbf2dc6b056 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:20.989598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18cd67-515d-4f01-bafe-1dbf2dc6b056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:20.991723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18cd67-515d-4f01-bafe-1dbf2dc6b056 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.019477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1142/2274] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:20 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:21.023824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16223a69-01b1-41e2-a0d2-9346f45713e4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.034330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16223a69-01b1-41e2-a0d2-9346f45713e4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.094800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640299df-e418-4ddb-bc6b-1a3c410eca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.095418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640299df-e418-4ddb-bc6b-1a3c410eca06 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.095697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640299df-e418-4ddb-bc6b-1a3c410eca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.095966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640299df-e418-4ddb-bc6b-1a3c410eca06 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.096649 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640299df-e418-4ddb-bc6b-1a3c410eca06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.098574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640299df-e418-4ddb-bc6b-1a3c410eca06 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.122521 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1133/2275] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:21 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:21.126894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18cd67-515d-4f01-bafe-1dbf2dc6b056 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.137792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18cd67-515d-4f01-bafe-1dbf2dc6b056 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.142426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4e0e5-b0fd-4a8a-b342-65f15234a92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.143085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e0e5-b0fd-4a8a-b342-65f15234a92f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.143401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e0e5-b0fd-4a8a-b342-65f15234a92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.143679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e0e5-b0fd-4a8a-b342-65f15234a92f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.144409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4e0e5-b0fd-4a8a-b342-65f15234a92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.146017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4e0e5-b0fd-4a8a-b342-65f15234a92f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.156585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1143/2276] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:21.160752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640299df-e418-4ddb-bc6b-1a3c410eca06 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.171393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640299df-e418-4ddb-bc6b-1a3c410eca06 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.176697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49318608-f15f-4744-a523-3dcb152e8059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.177347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49318608-f15f-4744-a523-3dcb152e8059 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.177627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49318608-f15f-4744-a523-3dcb152e8059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.177916 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49318608-f15f-4744-a523-3dcb152e8059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.178676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49318608-f15f-4744-a523-3dcb152e8059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.180425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49318608-f15f-4744-a523-3dcb152e8059 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.190571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1134/2277] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:21.194844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4e0e5-b0fd-4a8a-b342-65f15234a92f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.205014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4e0e5-b0fd-4a8a-b342-65f15234a92f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.209861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f7f7c-55d4-4b1a-b149-bf5fb6d83513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.210729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f7f7c-55d4-4b1a-b149-bf5fb6d83513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.211006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f7f7c-55d4-4b1a-b149-bf5fb6d83513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.211275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f7f7c-55d4-4b1a-b149-bf5fb6d83513 admin admin] PATH_INFO: `/v3/projects/6399b60aa0b949e9a588d291246302e4/users/3b703c16b59343c4a67763e0279d622a/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.226441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f7f7c-55d4-4b1a-b149-bf5fb6d83513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6399b60aa0b949e9a588d291246302e4, user_id=3b703c16b59343c4a67763e0279d622a, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.229401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f7f7c-55d4-4b1a-b149-bf5fb6d83513 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.243792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1144/2278] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:21 2022] PUT /identity/v3/projects/6399b60aa0b949e9a588d291246302e4/users/3b703c16b59343c4a67763e0279d622a/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:36:21.255420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90090f11-5041-47d2-9786-0714a2ef7bd9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.255717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90090f11-5041-47d2-9786-0714a2ef7bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.256001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90090f11-5041-47d2-9786-0714a2ef7bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.284952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-90090f11-5041-47d2-9786-0714a2ef7bd9 None None] MFA Rules not processed for user `3b703c16b59343c4a67763e0279d622a`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:21.286168 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-90090f11-5041-47d2-9786-0714a2ef7bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:21.314715 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:21.361638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1135/2279] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:21.366397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f7f7c-55d4-4b1a-b149-bf5fb6d83513 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.377101 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:21.431729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f7f7c-55d4-4b1a-b149-bf5fb6d83513 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.436896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eaa8b-dfa0-42a9-ad17-4a066d7f7961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.437563 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eaa8b-dfa0-42a9-ad17-4a066d7f7961 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.437842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eaa8b-dfa0-42a9-ad17-4a066d7f7961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.438119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eaa8b-dfa0-42a9-ad17-4a066d7f7961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.438799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eaa8b-dfa0-42a9-ad17-4a066d7f7961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.440781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eaa8b-dfa0-42a9-ad17-4a066d7f7961 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.472121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1145/2280] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:21 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:21.478072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90090f11-5041-47d2-9786-0714a2ef7bd9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.489557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90090f11-5041-47d2-9786-0714a2ef7bd9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.546657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6867188-6b76-4635-98a6-a84dc3152564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.547362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6867188-6b76-4635-98a6-a84dc3152564 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.547646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6867188-6b76-4635-98a6-a84dc3152564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.547918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6867188-6b76-4635-98a6-a84dc3152564 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.548601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6867188-6b76-4635-98a6-a84dc3152564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.550557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6867188-6b76-4635-98a6-a84dc3152564 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.571621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1136/2281] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:21 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:21.575670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eaa8b-dfa0-42a9-ad17-4a066d7f7961 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.586091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eaa8b-dfa0-42a9-ad17-4a066d7f7961 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.591595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d60b45-3834-4e8d-8204-f221f1ba96f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.592268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d60b45-3834-4e8d-8204-f221f1ba96f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.592550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d60b45-3834-4e8d-8204-f221f1ba96f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.592828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d60b45-3834-4e8d-8204-f221f1ba96f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.593560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d60b45-3834-4e8d-8204-f221f1ba96f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.595181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d60b45-3834-4e8d-8204-f221f1ba96f1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.606892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1146/2282] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:21.611331 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6867188-6b76-4635-98a6-a84dc3152564 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.622431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6867188-6b76-4635-98a6-a84dc3152564 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.628574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682681f8-b20d-4bb8-9bae-04845b3e37ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.629442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682681f8-b20d-4bb8-9bae-04845b3e37ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.629711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682681f8-b20d-4bb8-9bae-04845b3e37ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.629983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682681f8-b20d-4bb8-9bae-04845b3e37ad admin admin] PATH_INFO: `/v3/projects/1c8e0fbb53724994a4dac95939ac5b9b/users/160f3dcfe9ca48369525f40c1dc23de1/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.650020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682681f8-b20d-4bb8-9bae-04845b3e37ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c8e0fbb53724994a4dac95939ac5b9b, user_id=160f3dcfe9ca48369525f40c1dc23de1, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.652874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682681f8-b20d-4bb8-9bae-04845b3e37ad admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.668919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1137/2283] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:21 2022] PUT /identity/v3/projects/1c8e0fbb53724994a4dac95939ac5b9b/users/160f3dcfe9ca48369525f40c1dc23de1/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:21.673662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d60b45-3834-4e8d-8204-f221f1ba96f1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.685902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d60b45-3834-4e8d-8204-f221f1ba96f1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.749079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c901a3d-fd8d-4450-bba0-3a675ca6644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.749755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c901a3d-fd8d-4450-bba0-3a675ca6644c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.750035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c901a3d-fd8d-4450-bba0-3a675ca6644c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.750308 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c901a3d-fd8d-4450-bba0-3a675ca6644c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.751059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c901a3d-fd8d-4450-bba0-3a675ca6644c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.752731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c901a3d-fd8d-4450-bba0-3a675ca6644c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.766108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1147/2284] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:21.767534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682681f8-b20d-4bb8-9bae-04845b3e37ad admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.778163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682681f8-b20d-4bb8-9bae-04845b3e37ad admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.782765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11393ca-3244-40ef-90d8-0712c2a76bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.783670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11393ca-3244-40ef-90d8-0712c2a76bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.783952 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11393ca-3244-40ef-90d8-0712c2a76bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.784227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11393ca-3244-40ef-90d8-0712c2a76bc2 admin admin] PATH_INFO: `/v3/projects/1c8e0fbb53724994a4dac95939ac5b9b/users/160f3dcfe9ca48369525f40c1dc23de1/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.786181 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11393ca-3244-40ef-90d8-0712c2a76bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c8e0fbb53724994a4dac95939ac5b9b, user_id=160f3dcfe9ca48369525f40c1dc23de1, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:21.788874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11393ca-3244-40ef-90d8-0712c2a76bc2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:21.799436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1138/2285] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:21 2022] PUT /identity/v3/projects/1c8e0fbb53724994a4dac95939ac5b9b/users/160f3dcfe9ca48369525f40c1dc23de1/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:21.809535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f214-e976-4877-b808-5c1c3ffef75d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.809825 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f214-e976-4877-b808-5c1c3ffef75d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.810105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f214-e976-4877-b808-5c1c3ffef75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:21.841272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4ce1f214-e976-4877-b808-5c1c3ffef75d None None] MFA Rules not processed for user `160f3dcfe9ca48369525f40c1dc23de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:21.842588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4ce1f214-e976-4877-b808-5c1c3ffef75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:21.915068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1148/2286] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:36:21.926658 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11393ca-3244-40ef-90d8-0712c2a76bc2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:21.928604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b11393ca-3244-40ef-90d8-0712c2a76bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:21.986821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11393ca-3244-40ef-90d8-0712c2a76bc2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:21.990843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0c864-5a1b-4051-a2c7-ecf4d1ab136b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:21.991524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0c864-5a1b-4051-a2c7-ecf4d1ab136b service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:21.991810 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0c864-5a1b-4051-a2c7-ecf4d1ab136b service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:21.992078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0c864-5a1b-4051-a2c7-ecf4d1ab136b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:22.001655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0c864-5a1b-4051-a2c7-ecf4d1ab136b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:22.003082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0c864-5a1b-4051-a2c7-ecf4d1ab136b service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:22.012868 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1139/2287] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:22.413676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1f214-e976-4877-b808-5c1c3ffef75d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:22.419745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:22.491740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1f214-e976-4877-b808-5c1c3ffef75d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:22.497421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:22.498348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:22.498626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:22.499731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] PATH_INFO: `/v3/users/3b703c16b59343c4a67763e0279d622a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:22.501158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b703c16b59343c4a67763e0279d622a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:22.504280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:22.558488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b703c16b59343c4a67763e0279d622a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:22.559254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b703c16b59343c4a67763e0279d622a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:22.568283 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b703c16b59343c4a67763e0279d622a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:22.586250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b703c16b59343c4a67763e0279d622a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:22.587119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b703c16b59343c4a67763e0279d622a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:22.598978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1149/2288] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:22 2022] DELETE /identity/v3/users/3b703c16b59343c4a67763e0279d622a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:22.604917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ace828-d460-4cee-92cd-b4feb0696243 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:22.605379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ace828-d460-4cee-92cd-b4feb0696243 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:22.605501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ace828-d460-4cee-92cd-b4feb0696243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:22.624418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-60ace828-d460-4cee-92cd-b4feb0696243 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:22.625622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-60ace828-d460-4cee-92cd-b4feb0696243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:22.642233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:22.688520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1140/2289] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:22.698644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:22.700636 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:22.762529 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2a1bc-7d2c-412d-a54a-877986ac0163 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:22.766306 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e5ae-ecc7-401c-be68-5f373ce78331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:22.766977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e5ae-ecc7-401c-be68-5f373ce78331 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:22.767259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e5ae-ecc7-401c-be68-5f373ce78331 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:22.767565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e5ae-ecc7-401c-be68-5f373ce78331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:22.779135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950e5ae-ecc7-401c-be68-5f373ce78331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:22.780719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950e5ae-ecc7-401c-be68-5f373ce78331 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:22.791468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1150/2290] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:23.090938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ace828-d460-4cee-92cd-b4feb0696243 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:23.092546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-60ace828-d460-4cee-92cd-b4feb0696243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:23.105314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ace828-d460-4cee-92cd-b4feb0696243 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:23.110453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:23.111329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:23.111657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:23.112095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] PATH_INFO: `/v3/projects/6399b60aa0b949e9a588d291246302e4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:23.113389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6399b60aa0b949e9a588d291246302e4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:23.115342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:23.165126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6399b60aa0b949e9a588d291246302e4', 'request_id': 'req-a85d3e82-6ae2-421a-be48-fe3278d96b31'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.172851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6399b60aa0b949e9a588d291246302e4', 'request_id': 'req-a85d3e82-6ae2-421a-be48-fe3278d96b31'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.179607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6399b60aa0b949e9a588d291246302e4', 'request_id': 'req-a85d3e82-6ae2-421a-be48-fe3278d96b31'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.191357 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] The token cache is being invalidate because project 6399b60aa0b949e9a588d291246302e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:23.191742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.192583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.194320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1141/2291] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:23 2022] DELETE /identity/v3/projects/6399b60aa0b949e9a588d291246302e4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:23.198772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e5ae-ecc7-401c-be68-5f373ce78331 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:23.208503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e5ae-ecc7-401c-be68-5f373ce78331 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:23.233997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:23.268155 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:23.269040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:23.269320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:23.269586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] PATH_INFO: `/v3/users/160f3dcfe9ca48369525f40c1dc23de1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:23.270804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] RBAC: Authorizing `identity:delete_user(user_id=160f3dcfe9ca48369525f40c1dc23de1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:23.272804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:23.320808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '160f3dcfe9ca48369525f40c1dc23de1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.321609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '160f3dcfe9ca48369525f40c1dc23de1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.330045 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '160f3dcfe9ca48369525f40c1dc23de1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.342895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '160f3dcfe9ca48369525f40c1dc23de1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.343564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '160f3dcfe9ca48369525f40c1dc23de1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:23.356660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1151/2292] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:23 2022] DELETE /identity/v3/users/160f3dcfe9ca48369525f40c1dc23de1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:23.768742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:23.770799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:23.790015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d3e82-6ae2-421a-be48-fe3278d96b31 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:23.904988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:23.906183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:23.906626 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:23.907003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] PATH_INFO: `/v3/projects/1c8e0fbb53724994a4dac95939ac5b9b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:23.908785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c8e0fbb53724994a4dac95939ac5b9b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:23.911590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:23.913180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:23.924696 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:23.951815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79acd1-8a71-42fd-8b8c-9e262612a872 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:23.963548 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:23.965827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:23.966390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:23.966881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] PATH_INFO: `/v3/users/e2be7ad1d0a84ca996df5c80eae7506a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:23.968471 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2be7ad1d0a84ca996df5c80eae7506a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:23.970818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:24.044588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:24.046969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c8e0fbb53724994a4dac95939ac5b9b', 'request_id': 'req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.053965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2be7ad1d0a84ca996df5c80eae7506a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.056436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2be7ad1d0a84ca996df5c80eae7506a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.065088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c8e0fbb53724994a4dac95939ac5b9b', 'request_id': 'req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.076966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2be7ad1d0a84ca996df5c80eae7506a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.080645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c8e0fbb53724994a4dac95939ac5b9b', 'request_id': 'req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.114654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2be7ad1d0a84ca996df5c80eae7506a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.115511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a249d843-21ae-4500-8ed1-c14fbaf74a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2be7ad1d0a84ca996df5c80eae7506a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.119634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] The token cache is being invalidate because project 1c8e0fbb53724994a4dac95939ac5b9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:24.119939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.122442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-87d08d31-fa35-43f2-9ee5-7bc10f9ab458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:24.127805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1142/2293] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:23 2022] DELETE /identity/v3/projects/1c8e0fbb53724994a4dac95939ac5b9b => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:24.142961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8e505-5d80-421d-88f3-84f468fead40 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.143492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8e505-5d80-421d-88f3-84f468fead40 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.143835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8e505-5d80-421d-88f3-84f468fead40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.145626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1152/2294] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:23 2022] DELETE /identity/v3/users/e2be7ad1d0a84ca996df5c80eae7506a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:24.152368 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14608b72-01c8-47ee-8bee-53070fcc1d0a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.152777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14608b72-01c8-47ee-8bee-53070fcc1d0a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.153776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14608b72-01c8-47ee-8bee-53070fcc1d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.186509 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e1e8e505-5d80-421d-88f3-84f468fead40 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:24.192236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e1e8e505-5d80-421d-88f3-84f468fead40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:24.195258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-14608b72-01c8-47ee-8bee-53070fcc1d0a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:24.199771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-14608b72-01c8-47ee-8bee-53070fcc1d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:24.224500 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:24.288449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1143/2295] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:24.297202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97906d-1e03-4a48-a152-3ba169c493d8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.304325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97906d-1e03-4a48-a152-3ba169c493d8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.304615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97906d-1e03-4a48-a152-3ba169c493d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.306754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1153/2296] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:24.316907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14608b72-01c8-47ee-8bee-53070fcc1d0a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:24.318553 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-14608b72-01c8-47ee-8bee-53070fcc1d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:24.325509 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9d97906d-1e03-4a48-a152-3ba169c493d8 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:24.326866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9d97906d-1e03-4a48-a152-3ba169c493d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:24.342330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1144/2297] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:24.343526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97906d-1e03-4a48-a152-3ba169c493d8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:24.366816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97906d-1e03-4a48-a152-3ba169c493d8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:24.373151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1a1e-64f6-42e9-809d-cac59ad50ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:24.373911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1a1e-64f6-42e9-809d-cac59ad50ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.374283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1a1e-64f6-42e9-809d-cac59ad50ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.374874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce1a1e-64f6-42e9-809d-cac59ad50ccc admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.376567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce1a1e-64f6-42e9-809d-cac59ad50ccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:36:24.377001 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce1a1e-64f6-42e9-809d-cac59ad50ccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:24.378949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce1a1e-64f6-42e9-809d-cac59ad50ccc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:24.394692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1145/2298] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:36:24.399404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1a1e-64f6-42e9-809d-cac59ad50ccc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:24.413220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce1a1e-64f6-42e9-809d-cac59ad50ccc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:24.418486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f56b4-1672-435d-9ce2-a26ffdc1deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:24.419918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f56b4-1672-435d-9ce2-a26ffdc1deb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.420054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14608b72-01c8-47ee-8bee-53070fcc1d0a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:24.420321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f56b4-1672-435d-9ce2-a26ffdc1deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.420750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f56b4-1672-435d-9ce2-a26ffdc1deb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.421563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f56b4-1672-435d-9ce2-a26ffdc1deb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:24.423703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f56b4-1672-435d-9ce2-a26ffdc1deb7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:24.424265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fe26a-62bd-47e0-b97d-091d0927610e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:24.424987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fe26a-62bd-47e0-b97d-091d0927610e service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.425335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fe26a-62bd-47e0-b97d-091d0927610e service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.425675 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fe26a-62bd-47e0-b97d-091d0927610e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.436148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fe26a-62bd-47e0-b97d-091d0927610e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:24.437880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fe26a-62bd-47e0-b97d-091d0927610e service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:24.459494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1146/2299] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:24 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:24.460398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f56b4-1672-435d-9ce2-a26ffdc1deb7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:24.464159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1154/2300] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:36:24.550041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f56b4-1672-435d-9ce2-a26ffdc1deb7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:24.556596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6b6bb-7ea2-42e2-bcb0-ed635a211f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:24.557670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6b6bb-7ea2-42e2-bcb0-ed635a211f57 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.558234 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6b6bb-7ea2-42e2-bcb0-ed635a211f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.558567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6b6bb-7ea2-42e2-bcb0-ed635a211f57 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.559562 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6b6bb-7ea2-42e2-bcb0-ed635a211f57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:24.562121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6b6bb-7ea2-42e2-bcb0-ed635a211f57 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:24.598711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1147/2301] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:24 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:24.602670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fe26a-62bd-47e0-b97d-091d0927610e service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:24.618737 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fe26a-62bd-47e0-b97d-091d0927610e service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:24.639513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ab891-cc7e-450f-8c29-7a9d55c04661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:24.640674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ab891-cc7e-450f-8c29-7a9d55c04661 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.640959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ab891-cc7e-450f-8c29-7a9d55c04661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.641234 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ab891-cc7e-450f-8c29-7a9d55c04661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.641967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ab891-cc7e-450f-8c29-7a9d55c04661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:24.644115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ab891-cc7e-450f-8c29-7a9d55c04661 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:24.666899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1155/2302] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:24.676457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6b6bb-7ea2-42e2-bcb0-ed635a211f57 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:24.692462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6b6bb-7ea2-42e2-bcb0-ed635a211f57 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:24.705226 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9895e5f-7c71-49bf-8339-4d9b90550253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:24.706340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9895e5f-7c71-49bf-8339-4d9b90550253 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.708598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9895e5f-7c71-49bf-8339-4d9b90550253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.709132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9895e5f-7c71-49bf-8339-4d9b90550253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.711142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9895e5f-7c71-49bf-8339-4d9b90550253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:24.716666 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9895e5f-7c71-49bf-8339-4d9b90550253 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:24.742581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ab891-cc7e-450f-8c29-7a9d55c04661 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:24.743117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1148/2303] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:24.757743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ab891-cc7e-450f-8c29-7a9d55c04661 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:24.767332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059453b-3306-40fb-bce3-2a8f344c310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:24.769557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059453b-3306-40fb-bce3-2a8f344c310a admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.769923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059453b-3306-40fb-bce3-2a8f344c310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.770848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059453b-3306-40fb-bce3-2a8f344c310a admin admin] PATH_INFO: `/v3/projects/181dc870933246c2bdf7aaafd88e666b/users/a6e3f7778ed94af890054013579e8f1c/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.809416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059453b-3306-40fb-bce3-2a8f344c310a admin admin] RBAC: Authorizing `identity:create_grant(project_id=181dc870933246c2bdf7aaafd88e666b, user_id=a6e3f7778ed94af890054013579e8f1c, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:24.812682 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059453b-3306-40fb-bce3-2a8f344c310a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:24.851651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e89c2-f4ed-4ce2-834c-f238b9f4ea77 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:24.852469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e89c2-f4ed-4ce2-834c-f238b9f4ea77 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:24.852469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713e89c2-f4ed-4ce2-834c-f238b9f4ea77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:24.854662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1156/2304] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:24 2022] PUT /identity/v3/projects/181dc870933246c2bdf7aaafd88e666b/users/a6e3f7778ed94af890054013579e8f1c/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:24.880431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:24.885232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-713e89c2-f4ed-4ce2-834c-f238b9f4ea77 None None] MFA Rules not processed for user `a6e3f7778ed94af890054013579e8f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:24.886979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-713e89c2-f4ed-4ce2-834c-f238b9f4ea77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:24.969316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059453b-3306-40fb-bce3-2a8f344c310a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:24.972840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1059453b-3306-40fb-bce3-2a8f344c310a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:24.994181 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1149/2305] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:25.011117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e89c2-f4ed-4ce2-834c-f238b9f4ea77 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:25.014751 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-713e89c2-f4ed-4ce2-834c-f238b9f4ea77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:25.109296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713e89c2-f4ed-4ce2-834c-f238b9f4ea77 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:25.113643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7aabb-9529-4993-b59d-68d6992cecf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:25.114612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7aabb-9529-4993-b59d-68d6992cecf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:25.114735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7aabb-9529-4993-b59d-68d6992cecf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:25.114976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7aabb-9529-4993-b59d-68d6992cecf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:25.131514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:25.163494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7aabb-9529-4993-b59d-68d6992cecf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:25.165461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7aabb-9529-4993-b59d-68d6992cecf8 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:25.183335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1150/2306] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:25.212426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059453b-3306-40fb-bce3-2a8f344c310a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:25.217968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:25.218901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:25.219823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:25.220109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] PATH_INFO: `/v3/projects/e7196b49b5764ab3b33484f5cf311747` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:25.221862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7196b49b5764ab3b33484f5cf311747)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:25.225385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:25.316002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7196b49b5764ab3b33484f5cf311747', 'request_id': 'req-26777a7b-7e75-453e-a7a1-32c9db5beccc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.323523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7196b49b5764ab3b33484f5cf311747', 'request_id': 'req-26777a7b-7e75-453e-a7a1-32c9db5beccc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.340666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7196b49b5764ab3b33484f5cf311747', 'request_id': 'req-26777a7b-7e75-453e-a7a1-32c9db5beccc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.376808 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] The token cache is being invalidate because project e7196b49b5764ab3b33484f5cf311747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:25.379025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.381192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.384073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1157/2307] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:24 2022] DELETE /identity/v3/projects/e7196b49b5764ab3b33484f5cf311747 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:25.389114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7aabb-9529-4993-b59d-68d6992cecf8 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:25.405172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7aabb-9529-4993-b59d-68d6992cecf8 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:25.511688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:25.512977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:25.513410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:25.513864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] PATH_INFO: `/v3/users/fc5883e3dedc4e48a34f9fc5a4568ef5` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:25.515262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5883e3dedc4e48a34f9fc5a4568ef5)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:25.517852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:25.561501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:25.617918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5883e3dedc4e48a34f9fc5a4568ef5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.618629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5883e3dedc4e48a34f9fc5a4568ef5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.632920 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5883e3dedc4e48a34f9fc5a4568ef5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.633494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5883e3dedc4e48a34f9fc5a4568ef5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.649726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-72a42029-c2d2-4f44-9197-3086c62b383f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5883e3dedc4e48a34f9fc5a4568ef5'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:25.663942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1151/2308] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:25 2022] DELETE /identity/v3/users/fc5883e3dedc4e48a34f9fc5a4568ef5 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:26.159723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:26.161259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:26.176396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26777a7b-7e75-453e-a7a1-32c9db5beccc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:26.268878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:26.269745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.270279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.270680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] PATH_INFO: `/v3/projects/31efd41af2344fd5b3d5646f0c5d0126` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.274426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31efd41af2344fd5b3d5646f0c5d0126)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:26.274426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:26.344312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31efd41af2344fd5b3d5646f0c5d0126', 'request_id': 'req-9455f972-f96a-4f6f-aeef-a890158d79c6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:26.350156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:26.353393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31efd41af2344fd5b3d5646f0c5d0126', 'request_id': 'req-9455f972-f96a-4f6f-aeef-a890158d79c6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:26.366237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31efd41af2344fd5b3d5646f0c5d0126', 'request_id': 'req-9455f972-f96a-4f6f-aeef-a890158d79c6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:26.384024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] The token cache is being invalidate because project 31efd41af2344fd5b3d5646f0c5d0126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:26.384024 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:26.384718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9455f972-f96a-4f6f-aeef-a890158d79c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:26.386432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1158/2309] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:26 2022] DELETE /identity/v3/projects/31efd41af2344fd5b3d5646f0c5d0126 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:26.392813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef5321-d151-4ad5-8709-8fea5245a42b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.393345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef5321-d151-4ad5-8709-8fea5245a42b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.393492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef5321-d151-4ad5-8709-8fea5245a42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.445539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0fef5321-d151-4ad5-8709-8fea5245a42b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:26.447061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0fef5321-d151-4ad5-8709-8fea5245a42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:26.514486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1152/2310] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:26.517313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf2660-d9fa-48fd-83ce-351921613260 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.517594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf2660-d9fa-48fd-83ce-351921613260 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.517866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf2660-d9fa-48fd-83ce-351921613260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.537507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-70bf2660-d9fa-48fd-83ce-351921613260 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:26.538707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-70bf2660-d9fa-48fd-83ce-351921613260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:26.548061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1159/2311] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:26.552380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef5321-d151-4ad5-8709-8fea5245a42b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:26.562875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef5321-d151-4ad5-8709-8fea5245a42b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:26.569166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da1d64-f5d2-46ee-adef-c52406284c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:26.569825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da1d64-f5d2-46ee-adef-c52406284c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.570108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da1d64-f5d2-46ee-adef-c52406284c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.570377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da1d64-f5d2-46ee-adef-c52406284c4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.571313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da1d64-f5d2-46ee-adef-c52406284c4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:36:26.571713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da1d64-f5d2-46ee-adef-c52406284c4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:26.573324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da1d64-f5d2-46ee-adef-c52406284c4e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:26.585594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1153/2312] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:36:26.590166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf2660-d9fa-48fd-83ce-351921613260 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:26.599948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf2660-d9fa-48fd-83ce-351921613260 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:26.606227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838565a1-cf7c-405e-bb4e-68a682013405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:26.606907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838565a1-cf7c-405e-bb4e-68a682013405 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.607231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838565a1-cf7c-405e-bb4e-68a682013405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.607569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838565a1-cf7c-405e-bb4e-68a682013405 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.608257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838565a1-cf7c-405e-bb4e-68a682013405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:26.610159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838565a1-cf7c-405e-bb4e-68a682013405 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:26.635653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1160/2313] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:26 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:26.639886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da1d64-f5d2-46ee-adef-c52406284c4e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:26.650238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da1d64-f5d2-46ee-adef-c52406284c4e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:26.709426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b90975-9b63-40d4-ac25-113c95e144d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:26.710093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b90975-9b63-40d4-ac25-113c95e144d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.710372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b90975-9b63-40d4-ac25-113c95e144d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.710648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b90975-9b63-40d4-ac25-113c95e144d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.711462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b90975-9b63-40d4-ac25-113c95e144d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:26.713402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b90975-9b63-40d4-ac25-113c95e144d5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:26.722699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:26.740571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1154/2314] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:26 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:36:26.744241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838565a1-cf7c-405e-bb4e-68a682013405 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:26.754618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838565a1-cf7c-405e-bb4e-68a682013405 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:26.760040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83362303-e7c6-4edc-9631-8373af72ce20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:26.760694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83362303-e7c6-4edc-9631-8373af72ce20 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.760979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83362303-e7c6-4edc-9631-8373af72ce20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.761277 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83362303-e7c6-4edc-9631-8373af72ce20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.761993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83362303-e7c6-4edc-9631-8373af72ce20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:26.763618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83362303-e7c6-4edc-9631-8373af72ce20 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:26.774345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1161/2315] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:26.778361 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b90975-9b63-40d4-ac25-113c95e144d5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:26.788719 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b90975-9b63-40d4-ac25-113c95e144d5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:26.794517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1456be-2f6b-46e5-8dcc-c50c291cc0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:26.795194 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1456be-2f6b-46e5-8dcc-c50c291cc0ab admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.795508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1456be-2f6b-46e5-8dcc-c50c291cc0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.795781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1456be-2f6b-46e5-8dcc-c50c291cc0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.796483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1456be-2f6b-46e5-8dcc-c50c291cc0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:26.798075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1456be-2f6b-46e5-8dcc-c50c291cc0ab admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:26.808403 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1155/2316] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:26.812494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83362303-e7c6-4edc-9631-8373af72ce20 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:26.823829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83362303-e7c6-4edc-9631-8373af72ce20 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:26.828753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db510656-3338-4ea5-920f-5057c83993ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:26.829618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db510656-3338-4ea5-920f-5057c83993ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.829896 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db510656-3338-4ea5-920f-5057c83993ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.830167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db510656-3338-4ea5-920f-5057c83993ed admin admin] PATH_INFO: `/v3/projects/f1d978b23326438dbed31dfbf1600450/users/7bc51e7304d24a5ca737c063505873ae/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.845689 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db510656-3338-4ea5-920f-5057c83993ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d978b23326438dbed31dfbf1600450, user_id=7bc51e7304d24a5ca737c063505873ae, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:26.848416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db510656-3338-4ea5-920f-5057c83993ed admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:26.862923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1162/2317] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:26 2022] PUT /identity/v3/projects/f1d978b23326438dbed31dfbf1600450/users/7bc51e7304d24a5ca737c063505873ae/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:26.874231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4e1a3-1ecb-4123-9c0d-8c9e8ec9b29d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:26.874598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4e1a3-1ecb-4123-9c0d-8c9e8ec9b29d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:26.874945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4e1a3-1ecb-4123-9c0d-8c9e8ec9b29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:26.902329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-41f4e1a3-1ecb-4123-9c0d-8c9e8ec9b29d None None] MFA Rules not processed for user `7bc51e7304d24a5ca737c063505873ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:26.903682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-41f4e1a3-1ecb-4123-9c0d-8c9e8ec9b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:26.971936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1156/2318] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:26.984005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db510656-3338-4ea5-920f-5057c83993ed admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:26.986023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-db510656-3338-4ea5-920f-5057c83993ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:27.066990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db510656-3338-4ea5-920f-5057c83993ed admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:27.070969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a4ed17-f631-4450-9264-0e502a3b080a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:27.071794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4ed17-f631-4450-9264-0e502a3b080a service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:27.072163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4ed17-f631-4450-9264-0e502a3b080a service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:27.072547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a4ed17-f631-4450-9264-0e502a3b080a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:27.081419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4ed17-f631-4450-9264-0e502a3b080a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:27.082890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a4ed17-f631-4450-9264-0e502a3b080a service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:27.092170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1163/2319] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:27.116085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4e1a3-1ecb-4123-9c0d-8c9e8ec9b29d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:27.193881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4e1a3-1ecb-4123-9c0d-8c9e8ec9b29d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:27.198893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:27.199808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:27.200091 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:27.200364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] PATH_INFO: `/v3/users/7bc51e7304d24a5ca737c063505873ae` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:27.201567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bc51e7304d24a5ca737c063505873ae)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:27.203511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:27.250197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bc51e7304d24a5ca737c063505873ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.250839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bc51e7304d24a5ca737c063505873ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.259143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bc51e7304d24a5ca737c063505873ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.259909 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bc51e7304d24a5ca737c063505873ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.272096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bc51e7304d24a5ca737c063505873ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.274259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:27.288100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1157/2320] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:27 2022] DELETE /identity/v3/users/7bc51e7304d24a5ca737c063505873ae => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:27.292786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da0bf8-286a-474b-9ad0-02e32e7927b3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:27.293263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da0bf8-286a-474b-9ad0-02e32e7927b3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:27.293609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da0bf8-286a-474b-9ad0-02e32e7927b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:27.312252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-58da0bf8-286a-474b-9ad0-02e32e7927b3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:27.313372 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-58da0bf8-286a-474b-9ad0-02e32e7927b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:27.373418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1164/2321] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:36:27.384911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:27.386437 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:27.464413 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4a3f3-5b39-4368-953c-a3e12a69a2b2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:27.468373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2919b9-89a3-44f9-8f21-d670eb4cd2ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:27.469033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2919b9-89a3-44f9-8f21-d670eb4cd2ae service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:27.469311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2919b9-89a3-44f9-8f21-d670eb4cd2ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:27.469578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2919b9-89a3-44f9-8f21-d670eb4cd2ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:27.479051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2919b9-89a3-44f9-8f21-d670eb4cd2ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:27.480668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2919b9-89a3-44f9-8f21-d670eb4cd2ae service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:27.490275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1158/2322] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:27.863152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da0bf8-286a-474b-9ad0-02e32e7927b3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:27.864807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-58da0bf8-286a-474b-9ad0-02e32e7927b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:27.877597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da0bf8-286a-474b-9ad0-02e32e7927b3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:27.884497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:27.885388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:27.885663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:27.885938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] PATH_INFO: `/v3/projects/f1d978b23326438dbed31dfbf1600450` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:27.887185 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d978b23326438dbed31dfbf1600450)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:27.889245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:27.892642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:27.939948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d978b23326438dbed31dfbf1600450', 'request_id': 'req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.946483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d978b23326438dbed31dfbf1600450', 'request_id': 'req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.954113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d978b23326438dbed31dfbf1600450', 'request_id': 'req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.973246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] The token cache is being invalidate because project f1d978b23326438dbed31dfbf1600450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:27.973931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.975417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f5b0f3f4-1b36-4c48-9eac-3d23aa748a4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:27.977621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1165/2323] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:27 2022] DELETE /identity/v3/projects/f1d978b23326438dbed31dfbf1600450 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:27.983565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104bc1-d1ff-440d-a83d-e876589ca0e8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:27.983722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104bc1-d1ff-440d-a83d-e876589ca0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:27.983996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d104bc1-d1ff-440d-a83d-e876589ca0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.006839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5d104bc1-d1ff-440d-a83d-e876589ca0e8 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:28.008249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d104bc1-d1ff-440d-a83d-e876589ca0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:28.067927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1159/2324] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:28.076398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b785-574e-49bc-84e4-b1a4d580c24e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:28.076733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b785-574e-49bc-84e4-b1a4d580c24e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:28.078745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6b785-574e-49bc-84e4-b1a4d580c24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.097508 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7ad6b785-574e-49bc-84e4-b1a4d580c24e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:28.098770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7ad6b785-574e-49bc-84e4-b1a4d580c24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:28.112123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1166/2325] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:28.116891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d104bc1-d1ff-440d-a83d-e876589ca0e8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:28.128389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d104bc1-d1ff-440d-a83d-e876589ca0e8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:28.135172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3254b-c52c-4266-86bb-1c6c44b54bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:28.135877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3254b-c52c-4266-86bb-1c6c44b54bed admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:28.136192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3254b-c52c-4266-86bb-1c6c44b54bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:28.136537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3254b-c52c-4266-86bb-1c6c44b54bed admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.137259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3254b-c52c-4266-86bb-1c6c44b54bed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:36:28.137611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3254b-c52c-4266-86bb-1c6c44b54bed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:28.139279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3254b-c52c-4266-86bb-1c6c44b54bed admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:28.152125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1160/2326] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:36:28.156649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6b785-574e-49bc-84e4-b1a4d580c24e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:28.167506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6b785-574e-49bc-84e4-b1a4d580c24e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:28.172679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ee707-7ed0-4095-92c2-c864c287404a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:28.173373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ee707-7ed0-4095-92c2-c864c287404a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:28.173658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ee707-7ed0-4095-92c2-c864c287404a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:28.173929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ee707-7ed0-4095-92c2-c864c287404a admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.174660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ee707-7ed0-4095-92c2-c864c287404a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:28.176641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ee707-7ed0-4095-92c2-c864c287404a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:28.207321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1167/2327] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:28 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:28.209448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3254b-c52c-4266-86bb-1c6c44b54bed admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:28.221645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3254b-c52c-4266-86bb-1c6c44b54bed admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:28.291958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb45c8-2d6c-4876-be94-65184cf1c434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:28.292655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb45c8-2d6c-4876-be94-65184cf1c434 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:28.292945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb45c8-2d6c-4876-be94-65184cf1c434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:28.293223 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb45c8-2d6c-4876-be94-65184cf1c434 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.293933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb45c8-2d6c-4876-be94-65184cf1c434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:28.296243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb45c8-2d6c-4876-be94-65184cf1c434 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:28.306514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:28.325420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1161/2328] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:28 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:28.330229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ee707-7ed0-4095-92c2-c864c287404a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:28.352672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ee707-7ed0-4095-92c2-c864c287404a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:28.352672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef13be-83fa-4f8e-9fff-a94a2ece795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:28.352672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef13be-83fa-4f8e-9fff-a94a2ece795f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:28.352672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef13be-83fa-4f8e-9fff-a94a2ece795f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:28.353463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef13be-83fa-4f8e-9fff-a94a2ece795f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.353463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef13be-83fa-4f8e-9fff-a94a2ece795f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:28.354586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef13be-83fa-4f8e-9fff-a94a2ece795f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:28.374167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1168/2329] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:28.378115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb45c8-2d6c-4876-be94-65184cf1c434 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:28.388902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb45c8-2d6c-4876-be94-65184cf1c434 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:28.395113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70eaad-38ea-4d10-a443-0778d9e52414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:28.395886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70eaad-38ea-4d10-a443-0778d9e52414 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:28.396249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70eaad-38ea-4d10-a443-0778d9e52414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:28.396589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70eaad-38ea-4d10-a443-0778d9e52414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.397381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70eaad-38ea-4d10-a443-0778d9e52414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:28.399766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70eaad-38ea-4d10-a443-0778d9e52414 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:28.411332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1162/2330] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:36:28.415634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef13be-83fa-4f8e-9fff-a94a2ece795f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:28.426229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef13be-83fa-4f8e-9fff-a94a2ece795f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:28.431265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de4ce1-80a8-4c20-a7a7-32f34a78ad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:28.432255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de4ce1-80a8-4c20-a7a7-32f34a78ad5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:28.432611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de4ce1-80a8-4c20-a7a7-32f34a78ad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:28.433000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de4ce1-80a8-4c20-a7a7-32f34a78ad5d admin admin] PATH_INFO: `/v3/projects/b7f4fefc157246f09403ff6bab18641a/users/53e1a907daf54c34a5b3225d53ac0b56/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.449452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de4ce1-80a8-4c20-a7a7-32f34a78ad5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f4fefc157246f09403ff6bab18641a, user_id=53e1a907daf54c34a5b3225d53ac0b56, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:28.452264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de4ce1-80a8-4c20-a7a7-32f34a78ad5d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:28.468120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1169/2331] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:28 2022] PUT /identity/v3/projects/b7f4fefc157246f09403ff6bab18641a/users/53e1a907daf54c34a5b3225d53ac0b56/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:36:28.538026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d0178-763b-47ba-b21e-8241683ee400 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:28.538606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d0178-763b-47ba-b21e-8241683ee400 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:28.538735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d0178-763b-47ba-b21e-8241683ee400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.588942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dc0d0178-763b-47ba-b21e-8241683ee400 None None] MFA Rules not processed for user `53e1a907daf54c34a5b3225d53ac0b56`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:28.590602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dc0d0178-763b-47ba-b21e-8241683ee400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:28.655902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1163/2332] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:28.666939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de4ce1-80a8-4c20-a7a7-32f34a78ad5d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:28.668813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-55de4ce1-80a8-4c20-a7a7-32f34a78ad5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:28.730968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de4ce1-80a8-4c20-a7a7-32f34a78ad5d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:28.735508 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a1a2-d5b7-4502-b804-f595d37dc2f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:28.735689 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a1a2-d5b7-4502-b804-f595d37dc2f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:28.735980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a1a2-d5b7-4502-b804-f595d37dc2f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:28.736254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a1a2-d5b7-4502-b804-f595d37dc2f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:28.748294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3a1a2-d5b7-4502-b804-f595d37dc2f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:28.749767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3a1a2-d5b7-4502-b804-f595d37dc2f1 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:28.760226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1170/2333] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:34.659969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d0178-763b-47ba-b21e-8241683ee400 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:34.661428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dc0d0178-763b-47ba-b21e-8241683ee400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:34.733498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d0178-763b-47ba-b21e-8241683ee400 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:34.738783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:34.740335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:34.740767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:34.741227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] PATH_INFO: `/v3/users/01a61d25cce84864856f26a7dee55a1e` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:34.742653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a61d25cce84864856f26a7dee55a1e)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:34.745487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:34.799965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a61d25cce84864856f26a7dee55a1e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:34.801660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a61d25cce84864856f26a7dee55a1e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:34.811322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a61d25cce84864856f26a7dee55a1e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:34.812041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a61d25cce84864856f26a7dee55a1e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:34.830712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a61d25cce84864856f26a7dee55a1e'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:34.833158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:34.851389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1164/2334] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:34 2022] DELETE /identity/v3/users/01a61d25cce84864856f26a7dee55a1e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:34.854993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fc485-4946-476f-8ec8-1d47b320da62 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:34.855566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fc485-4946-476f-8ec8-1d47b320da62 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:34.856441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fc485-4946-476f-8ec8-1d47b320da62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:34.876347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-315fc485-4946-476f-8ec8-1d47b320da62 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:34.876347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-315fc485-4946-476f-8ec8-1d47b320da62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:34.903818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:34.943335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1171/2335] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:34.951094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:34.952631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:35.010158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1614a3-b618-493c-96b2-e25e648dead7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:35.014404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5307831-c261-4f47-bd9b-bfadfef96114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:35.015048 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5307831-c261-4f47-bd9b-bfadfef96114 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:35.015351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5307831-c261-4f47-bd9b-bfadfef96114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:35.015629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5307831-c261-4f47-bd9b-bfadfef96114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:35.024770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5307831-c261-4f47-bd9b-bfadfef96114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:35.026584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5307831-c261-4f47-bd9b-bfadfef96114 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:35.036664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1165/2336] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:35.317539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fc485-4946-476f-8ec8-1d47b320da62 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:35.319171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-315fc485-4946-476f-8ec8-1d47b320da62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:35.334299 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315fc485-4946-476f-8ec8-1d47b320da62 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:35.348739 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:35.349686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:35.349971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:35.350300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] PATH_INFO: `/v3/projects/54afb43716fa487bb7b8e3cfcea65a01` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:35.353257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=54afb43716fa487bb7b8e3cfcea65a01)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:35.356388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:35.417385 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54afb43716fa487bb7b8e3cfcea65a01', 'request_id': 'req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:35.424202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54afb43716fa487bb7b8e3cfcea65a01', 'request_id': 'req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:35.430856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54afb43716fa487bb7b8e3cfcea65a01', 'request_id': 'req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:35.445761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] The token cache is being invalidate because project 54afb43716fa487bb7b8e3cfcea65a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:35.446049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:35.446750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0618d9a-0dfa-42ed-94e6-6a5d73d42a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:35.448429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1172/2337] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:35 2022] DELETE /identity/v3/projects/54afb43716fa487bb7b8e3cfcea65a01 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:40.240380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5307831-c261-4f47-bd9b-bfadfef96114 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:40.242069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d5307831-c261-4f47-bd9b-bfadfef96114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:40.255346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5307831-c261-4f47-bd9b-bfadfef96114 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:40.318597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:40.319578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:40.319959 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:40.320238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] PATH_INFO: `/v3/users/53e1a907daf54c34a5b3225d53ac0b56` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:40.321541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e1a907daf54c34a5b3225d53ac0b56)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:40.323861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:40.378272 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e1a907daf54c34a5b3225d53ac0b56'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:40.379122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e1a907daf54c34a5b3225d53ac0b56'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:40.389024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e1a907daf54c34a5b3225d53ac0b56'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:40.389657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e1a907daf54c34a5b3225d53ac0b56'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:40.404272 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e1a907daf54c34a5b3225d53ac0b56'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:40.417604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1166/2338] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:40 2022] DELETE /identity/v3/users/53e1a907daf54c34a5b3225d53ac0b56 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:40.422946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7272fc-ea77-4c79-ac38-bec2b63db9d5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:40.423467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7272fc-ea77-4c79-ac38-bec2b63db9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:40.423641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7272fc-ea77-4c79-ac38-bec2b63db9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:40.442706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bc7272fc-ea77-4c79-ac38-bec2b63db9d5 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:40.444000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bc7272fc-ea77-4c79-ac38-bec2b63db9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:40.503703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1173/2339] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:40.513235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:40.514760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:40.528896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:40.577795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9aa2b-cfa5-4e08-973f-1a2d75822fa1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:40.581652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb0367-ee0b-4681-a101-a84d4303c48a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:40.582302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb0367-ee0b-4681-a101-a84d4303c48a service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:40.582569 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb0367-ee0b-4681-a101-a84d4303c48a service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:40.582841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb0367-ee0b-4681-a101-a84d4303c48a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:40.592488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb0367-ee0b-4681-a101-a84d4303c48a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:40.593994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb0367-ee0b-4681-a101-a84d4303c48a service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:40.604293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1167/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:41.207126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7272fc-ea77-4c79-ac38-bec2b63db9d5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:41.208773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bc7272fc-ea77-4c79-ac38-bec2b63db9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:41.230867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7272fc-ea77-4c79-ac38-bec2b63db9d5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:41.236070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:41.236953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.237238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.237517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] PATH_INFO: `/v3/projects/b7f4fefc157246f09403ff6bab18641a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.238699 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f4fefc157246f09403ff6bab18641a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:41.240695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:41.293876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f4fefc157246f09403ff6bab18641a', 'request_id': 'req-d0256949-0279-4d61-8555-5599c139aa36'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:41.295847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:41.301445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f4fefc157246f09403ff6bab18641a', 'request_id': 'req-d0256949-0279-4d61-8555-5599c139aa36'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:41.308210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f4fefc157246f09403ff6bab18641a', 'request_id': 'req-d0256949-0279-4d61-8555-5599c139aa36'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:41.322473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] The token cache is being invalidate because project b7f4fefc157246f09403ff6bab18641a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:41.322735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:41.323844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d0256949-0279-4d61-8555-5599c139aa36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:41.327130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1174/2341] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:41 2022] DELETE /identity/v3/projects/b7f4fefc157246f09403ff6bab18641a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:41.334533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac3ba0-cc31-43e5-b53c-61057bcae36f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.335030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac3ba0-cc31-43e5-b53c-61057bcae36f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.335122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac3ba0-cc31-43e5-b53c-61057bcae36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.366597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-20ac3ba0-cc31-43e5-b53c-61057bcae36f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:41.367832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-20ac3ba0-cc31-43e5-b53c-61057bcae36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:41.431279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1168/2342] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:41.439853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01319fe5-d677-4caa-aca1-6f38e5396621 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.440421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01319fe5-d677-4caa-aca1-6f38e5396621 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.440421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01319fe5-d677-4caa-aca1-6f38e5396621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.460322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-01319fe5-d677-4caa-aca1-6f38e5396621 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:41.461591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-01319fe5-d677-4caa-aca1-6f38e5396621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:41.470469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1175/2343] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:41.475026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac3ba0-cc31-43e5-b53c-61057bcae36f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:41.485458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac3ba0-cc31-43e5-b53c-61057bcae36f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:41.491014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61daa8-59ad-44c9-926d-5b2b7e2c4b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:41.491702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61daa8-59ad-44c9-926d-5b2b7e2c4b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.491977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61daa8-59ad-44c9-926d-5b2b7e2c4b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.492257 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61daa8-59ad-44c9-926d-5b2b7e2c4b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.492982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61daa8-59ad-44c9-926d-5b2b7e2c4b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:36:41.493288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61daa8-59ad-44c9-926d-5b2b7e2c4b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:41.494900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61daa8-59ad-44c9-926d-5b2b7e2c4b35 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:41.506978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1169/2344] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:36:41.511382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01319fe5-d677-4caa-aca1-6f38e5396621 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:41.521244 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01319fe5-d677-4caa-aca1-6f38e5396621 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:41.527131 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008c8ec-7694-479a-839b-1a6deffb667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:41.527810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008c8ec-7694-479a-839b-1a6deffb667c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.528088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008c8ec-7694-479a-839b-1a6deffb667c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.528359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008c8ec-7694-479a-839b-1a6deffb667c admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.529117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008c8ec-7694-479a-839b-1a6deffb667c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:41.531169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008c8ec-7694-479a-839b-1a6deffb667c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:41.559938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1176/2345] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:41 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:41.564353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61daa8-59ad-44c9-926d-5b2b7e2c4b35 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:41.574645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61daa8-59ad-44c9-926d-5b2b7e2c4b35 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:41.638247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc6530-040c-4950-ab10-288c77186f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:41.638940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc6530-040c-4950-ab10-288c77186f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.640356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc6530-040c-4950-ab10-288c77186f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.640356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc6530-040c-4950-ab10-288c77186f51 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.640949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc6530-040c-4950-ab10-288c77186f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:41.643590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc6530-040c-4950-ab10-288c77186f51 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:41.668275 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1170/2346] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:41 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:36:41.673057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008c8ec-7694-479a-839b-1a6deffb667c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:41.688941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008c8ec-7694-479a-839b-1a6deffb667c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:41.693383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c726d6c-36bb-4a3b-b005-7a423570871c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:41.694059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c726d6c-36bb-4a3b-b005-7a423570871c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.694335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c726d6c-36bb-4a3b-b005-7a423570871c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.694607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c726d6c-36bb-4a3b-b005-7a423570871c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.695859 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c726d6c-36bb-4a3b-b005-7a423570871c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:41.697146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c726d6c-36bb-4a3b-b005-7a423570871c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:41.708782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1177/2347] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:36:41.713012 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc6530-040c-4950-ab10-288c77186f51 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:41.722977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc6530-040c-4950-ab10-288c77186f51 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:41.727523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76aa116-538d-496b-bde8-b4bd8a00d65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:41.728204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76aa116-538d-496b-bde8-b4bd8a00d65b admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.728486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76aa116-538d-496b-bde8-b4bd8a00d65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.728762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76aa116-538d-496b-bde8-b4bd8a00d65b admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.729515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76aa116-538d-496b-bde8-b4bd8a00d65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:41.731243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76aa116-538d-496b-bde8-b4bd8a00d65b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:41.741826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1171/2348] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:41.746107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c726d6c-36bb-4a3b-b005-7a423570871c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:41.757531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c726d6c-36bb-4a3b-b005-7a423570871c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:41.763544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b0a1b-91e5-4036-a57b-8dec019af408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:41.764446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b0a1b-91e5-4036-a57b-8dec019af408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.764726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b0a1b-91e5-4036-a57b-8dec019af408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.764995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b0a1b-91e5-4036-a57b-8dec019af408 admin admin] PATH_INFO: `/v3/projects/81a1413099324b758a2608fa5d67c0a3/users/662efed3aab046bfb378bbffc49a9c11/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.780229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b0a1b-91e5-4036-a57b-8dec019af408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a1413099324b758a2608fa5d67c0a3, user_id=662efed3aab046bfb378bbffc49a9c11, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:41.782890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b0a1b-91e5-4036-a57b-8dec019af408 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:41.797667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1178/2349] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:41 2022] PUT /identity/v3/projects/81a1413099324b758a2608fa5d67c0a3/users/662efed3aab046bfb378bbffc49a9c11/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:41.802783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7c5a2-cd9f-4145-98f4-937a26cd61b7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.803162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7c5a2-cd9f-4145-98f4-937a26cd61b7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.803539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7c5a2-cd9f-4145-98f4-937a26cd61b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.821473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9cd7c5a2-cd9f-4145-98f4-937a26cd61b7 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:41.822690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9cd7c5a2-cd9f-4145-98f4-937a26cd61b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:41.828005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:41.887022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1172/2350] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:41.897637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b0a1b-91e5-4036-a57b-8dec019af408 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:41.900265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-430b0a1b-91e5-4036-a57b-8dec019af408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:41.963611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b0a1b-91e5-4036-a57b-8dec019af408 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:41.967946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d654787-798b-493d-a5bd-b5b9e8387ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:41.968653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d654787-798b-493d-a5bd-b5b9e8387ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:41.968931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d654787-798b-493d-a5bd-b5b9e8387ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:41.969204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d654787-798b-493d-a5bd-b5b9e8387ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:41.978897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d654787-798b-493d-a5bd-b5b9e8387ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:41.980406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d654787-798b-493d-a5bd-b5b9e8387ac5 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:41.990552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1179/2351] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:42.646976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787888fb-0c90-4a96-b614-f315798cf472 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:42.647551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787888fb-0c90-4a96-b614-f315798cf472 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:42.647551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787888fb-0c90-4a96-b614-f315798cf472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:42.675444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-787888fb-0c90-4a96-b614-f315798cf472 None None] MFA Rules not processed for user `662efed3aab046bfb378bbffc49a9c11`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:42.676724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-787888fb-0c90-4a96-b614-f315798cf472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:42.757470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1173/2352] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:42.770737 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d654787-798b-493d-a5bd-b5b9e8387ac5 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:42.772334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4d654787-798b-493d-a5bd-b5b9e8387ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:42.839205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d654787-798b-493d-a5bd-b5b9e8387ac5 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:42.843439 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf563fba-d922-4d68-930d-6532d40e30e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:42.844125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf563fba-d922-4d68-930d-6532d40e30e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:42.844409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf563fba-d922-4d68-930d-6532d40e30e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:42.844694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf563fba-d922-4d68-930d-6532d40e30e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:42.853832 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf563fba-d922-4d68-930d-6532d40e30e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:42.855412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf563fba-d922-4d68-930d-6532d40e30e3 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:42.864910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1180/2353] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:47.392347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e634e-d927-476d-b1eb-697d1653393d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:47.393030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e634e-d927-476d-b1eb-697d1653393d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:47.393030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e634e-d927-476d-b1eb-697d1653393d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:47.413210 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-831e634e-d927-476d-b1eb-697d1653393d None None] MFA Rules not processed for user `662efed3aab046bfb378bbffc49a9c11`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:47.414442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-831e634e-d927-476d-b1eb-697d1653393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:47.422566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1174/2354] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:51.385981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf563fba-d922-4d68-930d-6532d40e30e3 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:51.387514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cf563fba-d922-4d68-930d-6532d40e30e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:51.405867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf563fba-d922-4d68-930d-6532d40e30e3 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:51.414584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:51.415499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:51.415719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:51.416022 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] PATH_INFO: `/v3/users/a6e3f7778ed94af890054013579e8f1c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:51.417430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6e3f7778ed94af890054013579e8f1c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:51.419957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:51.439105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:51.477712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6e3f7778ed94af890054013579e8f1c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:51.478367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6e3f7778ed94af890054013579e8f1c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:51.488202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6e3f7778ed94af890054013579e8f1c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:51.500162 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6e3f7778ed94af890054013579e8f1c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:51.500854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6e3f7778ed94af890054013579e8f1c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:51.513164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1181/2355] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:36:51 2022] DELETE /identity/v3/users/a6e3f7778ed94af890054013579e8f1c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:51.518446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73be69c-a0fa-43e5-ada0-2147c2a37f0a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:51.518975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73be69c-a0fa-43e5-ada0-2147c2a37f0a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:51.518975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73be69c-a0fa-43e5-ada0-2147c2a37f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:51.537243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e73be69c-a0fa-43e5-ada0-2147c2a37f0a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:51.538387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e73be69c-a0fa-43e5-ada0-2147c2a37f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:51.600384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1175/2356] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:51.610254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:51.611891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:51.674021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105d1b8-e519-49f8-b46a-40c86d6356c9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:51.678199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7c74b-da91-4c1e-85ec-e884956dd3da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:51.678849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7c74b-da91-4c1e-85ec-e884956dd3da service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:51.679114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7c74b-da91-4c1e-85ec-e884956dd3da service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:51.679473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7c74b-da91-4c1e-85ec-e884956dd3da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:51.689164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7c74b-da91-4c1e-85ec-e884956dd3da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:51.690607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7c74b-da91-4c1e-85ec-e884956dd3da service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:51.700622 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1182/2357] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:52.008847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73be69c-a0fa-43e5-ada0-2147c2a37f0a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:52.018865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73be69c-a0fa-43e5-ada0-2147c2a37f0a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:52.022869 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4d3fb-d5bd-4370-83e5-413130642be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:52.023548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d3fb-d5bd-4370-83e5-413130642be0 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.023842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d3fb-d5bd-4370-83e5-413130642be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.024126 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4d3fb-d5bd-4370-83e5-413130642be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.026313 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-55a4d3fb-d5bd-4370-83e5-413130642be0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:52.051003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7c74b-da91-4c1e-85ec-e884956dd3da service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:52.052588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c6b7c74b-da91-4c1e-85ec-e884956dd3da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:52.075167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7c74b-da91-4c1e-85ec-e884956dd3da service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:52.080016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:52.087517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:52.088395 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.088827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.089564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] PATH_INFO: `/v3/projects/181dc870933246c2bdf7aaafd88e666b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.090758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=181dc870933246c2bdf7aaafd88e666b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:52.094164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:52.123016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4d3fb-d5bd-4370-83e5-413130642be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:52.124913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4d3fb-d5bd-4370-83e5-413130642be0 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:52.137139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1176/2358] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:52.151577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181dc870933246c2bdf7aaafd88e666b', 'request_id': 'req-146360c9-a534-4748-8587-bf3c311c4090'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:52.158335 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181dc870933246c2bdf7aaafd88e666b', 'request_id': 'req-146360c9-a534-4748-8587-bf3c311c4090'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:52.165849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181dc870933246c2bdf7aaafd88e666b', 'request_id': 'req-146360c9-a534-4748-8587-bf3c311c4090'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:52.187769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] The token cache is being invalidate because project 181dc870933246c2bdf7aaafd88e666b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:36:52.188812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:52.189665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-146360c9-a534-4748-8587-bf3c311c4090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:36:52.191410 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1183/2359] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:36:52 2022] DELETE /identity/v3/projects/181dc870933246c2bdf7aaafd88e666b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:52.197833 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fa107-bb5e-45f4-9c33-ddf81a8d6913 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.198140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fa107-bb5e-45f4-9c33-ddf81a8d6913 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.198422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fa107-bb5e-45f4-9c33-ddf81a8d6913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.246473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7c7fa107-bb5e-45f4-9c33-ddf81a8d6913 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:52.247970 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7c7fa107-bb5e-45f4-9c33-ddf81a8d6913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:52.325118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1177/2360] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:52.333419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b0648-0af8-4c6d-9476-133bd841bb7b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.333709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b0648-0af8-4c6d-9476-133bd841bb7b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.333986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b0648-0af8-4c6d-9476-133bd841bb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.353819 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-395b0648-0af8-4c6d-9476-133bd841bb7b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:52.355340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-395b0648-0af8-4c6d-9476-133bd841bb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:52.363784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1184/2361] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:52.368018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fa107-bb5e-45f4-9c33-ddf81a8d6913 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:52.378346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fa107-bb5e-45f4-9c33-ddf81a8d6913 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:52.383584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b42d48-340d-413e-9db8-3d7102bed338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:52.384337 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b42d48-340d-413e-9db8-3d7102bed338 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.384668 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b42d48-340d-413e-9db8-3d7102bed338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.384937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b42d48-340d-413e-9db8-3d7102bed338 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.385655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b42d48-340d-413e-9db8-3d7102bed338 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:36:52.385958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b42d48-340d-413e-9db8-3d7102bed338 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:52.387575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b42d48-340d-413e-9db8-3d7102bed338 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:52.399726 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1178/2362] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:36:52.409757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b0648-0af8-4c6d-9476-133bd841bb7b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:52.419588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b0648-0af8-4c6d-9476-133bd841bb7b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:52.424627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46710371-e554-49b3-9229-c99b8794a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:52.425292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710371-e554-49b3-9229-c99b8794a22d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.425572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710371-e554-49b3-9229-c99b8794a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.425845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46710371-e554-49b3-9229-c99b8794a22d admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.426536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46710371-e554-49b3-9229-c99b8794a22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:52.428511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46710371-e554-49b3-9229-c99b8794a22d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:52.437626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:52.459120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1185/2363] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:36:52 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:52.463326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b42d48-340d-413e-9db8-3d7102bed338 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:52.480402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b42d48-340d-413e-9db8-3d7102bed338 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:52.577488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae42df-810d-4d50-9cae-17ce37824fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:52.578273 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae42df-810d-4d50-9cae-17ce37824fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.578389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae42df-810d-4d50-9cae-17ce37824fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.578586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae42df-810d-4d50-9cae-17ce37824fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.579278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae42df-810d-4d50-9cae-17ce37824fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:52.581274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae42df-810d-4d50-9cae-17ce37824fd6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:52.609775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1179/2364] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:36:52 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:36:52.611477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46710371-e554-49b3-9229-c99b8794a22d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:52.622485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46710371-e554-49b3-9229-c99b8794a22d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:52.629314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c09d72-e7a0-4430-a734-ec9d9925f393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:52.630237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c09d72-e7a0-4430-a734-ec9d9925f393 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.631238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c09d72-e7a0-4430-a734-ec9d9925f393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.631572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c09d72-e7a0-4430-a734-ec9d9925f393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.632348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c09d72-e7a0-4430-a734-ec9d9925f393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:52.633961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c09d72-e7a0-4430-a734-ec9d9925f393 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:52.648944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1186/2365] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:36:52.650110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae42df-810d-4d50-9cae-17ce37824fd6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:52.663639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae42df-810d-4d50-9cae-17ce37824fd6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:52.669874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19251b56-9da6-4638-a992-91be89afdb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:52.670549 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19251b56-9da6-4638-a992-91be89afdb27 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.670905 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19251b56-9da6-4638-a992-91be89afdb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.671186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19251b56-9da6-4638-a992-91be89afdb27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.672399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19251b56-9da6-4638-a992-91be89afdb27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:52.674054 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19251b56-9da6-4638-a992-91be89afdb27 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:52.685605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1180/2366] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:36:52.690023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c09d72-e7a0-4430-a734-ec9d9925f393 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:52.701286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c09d72-e7a0-4430-a734-ec9d9925f393 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:52.706985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa334a9-e33f-46e2-a711-198cd7fb93ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:52.708586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa334a9-e33f-46e2-a711-198cd7fb93ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.709027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa334a9-e33f-46e2-a711-198cd7fb93ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.709442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa334a9-e33f-46e2-a711-198cd7fb93ee admin admin] PATH_INFO: `/v3/projects/d54e257b987143cf91a2a902d37276af/users/302eefe4074b4dc5b1f967727236d971/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.729297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa334a9-e33f-46e2-a711-198cd7fb93ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54e257b987143cf91a2a902d37276af, user_id=302eefe4074b4dc5b1f967727236d971, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:52.732513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa334a9-e33f-46e2-a711-198cd7fb93ee admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:52.760048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1187/2367] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:36:52 2022] PUT /identity/v3/projects/d54e257b987143cf91a2a902d37276af/users/302eefe4074b4dc5b1f967727236d971/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:36:52.771449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc905d-fb51-454b-99ea-687ffa860ec0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:52.771742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc905d-fb51-454b-99ea-687ffa860ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:52.772165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc905d-fb51-454b-99ea-687ffa860ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:52.819073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9abc905d-fb51-454b-99ea-687ffa860ec0 None None] MFA Rules not processed for user `302eefe4074b4dc5b1f967727236d971`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:52.822364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9abc905d-fb51-454b-99ea-687ffa860ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:52.852786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:36:52.935448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1181/2368] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:36:52.943489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa334a9-e33f-46e2-a711-198cd7fb93ee admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:36:52.945498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-daa334a9-e33f-46e2-a711-198cd7fb93ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:53.004602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa334a9-e33f-46e2-a711-198cd7fb93ee admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:36:53.008581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775dce54-1590-425f-844d-470844228fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:36:53.009315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dce54-1590-425f-844d-470844228fac service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:53.009687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dce54-1590-425f-844d-470844228fac service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:53.010025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dce54-1590-425f-844d-470844228fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:53.019665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dce54-1590-425f-844d-470844228fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:36:53.021354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775dce54-1590-425f-844d-470844228fac service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:36:53.032668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1188/2369] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:36:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:36:59.513462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc283e-5234-4c4c-8d00-94b234af52a1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:36:59.514020 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc283e-5234-4c4c-8d00-94b234af52a1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:36:59.514122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc283e-5234-4c4c-8d00-94b234af52a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:36:59.533251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fdcc283e-5234-4c4c-8d00-94b234af52a1 None None] MFA Rules not processed for user `c941dc00e0584666aba13263b8c1fcb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:36:59.534475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fdcc283e-5234-4c4c-8d00-94b234af52a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:36:59.593592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1182/2370] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:36:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:37:03.027730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:37:09.586336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=41 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:37:20.722595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647019a-51d4-4ca9-b1c3-3c188c6e41b9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:20.723177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647019a-51d4-4ca9-b1c3-3c188c6e41b9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:20.723177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647019a-51d4-4ca9-b1c3-3c188c6e41b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:20.746143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c647019a-51d4-4ca9-b1c3-3c188c6e41b9 None None] MFA Rules not processed for user `662efed3aab046bfb378bbffc49a9c11`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:37:20.748075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c647019a-51d4-4ca9-b1c3-3c188c6e41b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:20.811495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1189/2371] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:37:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:37:21.933788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc283e-5234-4c4c-8d00-94b234af52a1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:21.935250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fdcc283e-5234-4c4c-8d00-94b234af52a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:22.036513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc283e-5234-4c4c-8d00-94b234af52a1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:22.042906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857c09f-9e2b-4a9a-89ec-35b845b9f47e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:22.043587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857c09f-9e2b-4a9a-89ec-35b845b9f47e service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:22.044219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857c09f-9e2b-4a9a-89ec-35b845b9f47e service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:22.044496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857c09f-9e2b-4a9a-89ec-35b845b9f47e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:22.046487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2857c09f-9e2b-4a9a-89ec-35b845b9f47e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:22.058813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857c09f-9e2b-4a9a-89ec-35b845b9f47e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:22.060513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857c09f-9e2b-4a9a-89ec-35b845b9f47e service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:22.070780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1183/2372] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:37:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:37:30.803554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=25 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:37:32.066567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=41 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:37:40.267853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647019a-51d4-4ca9-b1c3-3c188c6e41b9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:40.336849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647019a-51d4-4ca9-b1c3-3c188c6e41b9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:40.341614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:40.342489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:40.342768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:40.343041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] PATH_INFO: `/v3/users/302eefe4074b4dc5b1f967727236d971` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:40.344268 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=302eefe4074b4dc5b1f967727236d971)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:40.346264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:40.393661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302eefe4074b4dc5b1f967727236d971'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:40.394340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302eefe4074b4dc5b1f967727236d971'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:40.403554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302eefe4074b4dc5b1f967727236d971'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:40.416342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302eefe4074b4dc5b1f967727236d971'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:40.417017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302eefe4074b4dc5b1f967727236d971'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:40.430143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1190/2373] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:37:40 2022] DELETE /identity/v3/users/302eefe4074b4dc5b1f967727236d971 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:37:40.434871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed46e0d-1fde-4820-b337-67b0c062fdb3 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:40.435568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed46e0d-1fde-4820-b337-67b0c062fdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:40.435757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed46e0d-1fde-4820-b337-67b0c062fdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:40.455171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2ed46e0d-1fde-4820-b337-67b0c062fdb3 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:37:40.456405 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2ed46e0d-1fde-4820-b337-67b0c062fdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:40.514599 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1184/2374] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:37:40.524445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:40.525994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:40.588228 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b49a71-1869-43dc-9c92-d2c8d63b6407 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:40.595539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b93afa-6375-4592-855a-a6574413d700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:40.596383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b93afa-6375-4592-855a-a6574413d700 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:40.596728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b93afa-6375-4592-855a-a6574413d700 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:40.597005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b93afa-6375-4592-855a-a6574413d700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:40.608498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b93afa-6375-4592-855a-a6574413d700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:40.610000 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b93afa-6375-4592-855a-a6574413d700 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:40.620649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1191/2375] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:37:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:37:40.921870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed46e0d-1fde-4820-b337-67b0c062fdb3 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:40.923507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2ed46e0d-1fde-4820-b337-67b0c062fdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:40.938370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed46e0d-1fde-4820-b337-67b0c062fdb3 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:40.944249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:40.945925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:40.946447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:40.946800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] PATH_INFO: `/v3/projects/d54e257b987143cf91a2a902d37276af` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:40.948393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54e257b987143cf91a2a902d37276af)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:40.950558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:41.004735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54e257b987143cf91a2a902d37276af', 'request_id': 'req-930276bf-376c-44d1-b324-0f152f10f206'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:41.014326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54e257b987143cf91a2a902d37276af', 'request_id': 'req-930276bf-376c-44d1-b324-0f152f10f206'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:41.020790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54e257b987143cf91a2a902d37276af', 'request_id': 'req-930276bf-376c-44d1-b324-0f152f10f206'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:41.032289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] The token cache is being invalidate because project d54e257b987143cf91a2a902d37276af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:37:41.032576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:41.033216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-930276bf-376c-44d1-b324-0f152f10f206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:37:41.034747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1185/2376] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:37:40 2022] DELETE /identity/v3/projects/d54e257b987143cf91a2a902d37276af => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:37:41.039614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97898ee1-5984-416d-98ec-a6f18848129b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.039999 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97898ee1-5984-416d-98ec-a6f18848129b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.040283 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97898ee1-5984-416d-98ec-a6f18848129b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.058486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-97898ee1-5984-416d-98ec-a6f18848129b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:37:41.059980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-97898ee1-5984-416d-98ec-a6f18848129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:41.077655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:37:41.125265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1192/2377] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:37:41.134956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0571-62a0-4adf-9b78-07916600721a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.135355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0571-62a0-4adf-9b78-07916600721a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.135692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0571-62a0-4adf-9b78-07916600721a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.153975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6e7f0571-62a0-4adf-9b78-07916600721a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:37:41.155203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6e7f0571-62a0-4adf-9b78-07916600721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:41.163035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1186/2378] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:37:41.167602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97898ee1-5984-416d-98ec-a6f18848129b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:41.179029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97898ee1-5984-416d-98ec-a6f18848129b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:41.183836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7ca05-ccca-4fa9-8b3e-6a24f32f524b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:41.184481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7ca05-ccca-4fa9-8b3e-6a24f32f524b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.184787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7ca05-ccca-4fa9-8b3e-6a24f32f524b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.185260 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7ca05-ccca-4fa9-8b3e-6a24f32f524b admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.185966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7ca05-ccca-4fa9-8b3e-6a24f32f524b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:37:41.187582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7ca05-ccca-4fa9-8b3e-6a24f32f524b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:41.189380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c7ca05-ccca-4fa9-8b3e-6a24f32f524b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:41.202556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1193/2379] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:37:41.207128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f0571-62a0-4adf-9b78-07916600721a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:41.216642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f0571-62a0-4adf-9b78-07916600721a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:41.222246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788963aa-c4f8-4b2d-ac13-d1e8918b456a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:41.222900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788963aa-c4f8-4b2d-ac13-d1e8918b456a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.223179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788963aa-c4f8-4b2d-ac13-d1e8918b456a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.223716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788963aa-c4f8-4b2d-ac13-d1e8918b456a admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.224177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788963aa-c4f8-4b2d-ac13-d1e8918b456a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:41.226180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788963aa-c4f8-4b2d-ac13-d1e8918b456a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:41.260341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1187/2380] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:37:41 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:37:41.261693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7ca05-ccca-4fa9-8b3e-6a24f32f524b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:41.272717 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7ca05-ccca-4fa9-8b3e-6a24f32f524b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:41.334455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf6de1-8246-4a9f-9ae6-1494d8a2100f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:41.334878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf6de1-8246-4a9f-9ae6-1494d8a2100f admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.335141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf6de1-8246-4a9f-9ae6-1494d8a2100f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.335470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf6de1-8246-4a9f-9ae6-1494d8a2100f admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.336177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf6de1-8246-4a9f-9ae6-1494d8a2100f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:41.338267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf6de1-8246-4a9f-9ae6-1494d8a2100f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:41.365558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1194/2381] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:37:41 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:37:41.367281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788963aa-c4f8-4b2d-ac13-d1e8918b456a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:41.377451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788963aa-c4f8-4b2d-ac13-d1e8918b456a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:41.382305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d74282-f983-464f-a870-5d99313b3315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:41.382956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74282-f983-464f-a870-5d99313b3315 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.383232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74282-f983-464f-a870-5d99313b3315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.384142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d74282-f983-464f-a870-5d99313b3315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.384869 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d74282-f983-464f-a870-5d99313b3315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:41.386469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d74282-f983-464f-a870-5d99313b3315 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:41.397030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1188/2382] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:37:41.401109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf6de1-8246-4a9f-9ae6-1494d8a2100f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:41.411445 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf6de1-8246-4a9f-9ae6-1494d8a2100f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:41.416957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cd50b-5706-4d8d-9ae6-5530f9e92721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:41.417614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cd50b-5706-4d8d-9ae6-5530f9e92721 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.417899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cd50b-5706-4d8d-9ae6-5530f9e92721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.418175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cd50b-5706-4d8d-9ae6-5530f9e92721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.419220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cd50b-5706-4d8d-9ae6-5530f9e92721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:41.420854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cd50b-5706-4d8d-9ae6-5530f9e92721 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:41.430950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1195/2383] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:37:41.435059 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d74282-f983-464f-a870-5d99313b3315 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:41.445513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d74282-f983-464f-a870-5d99313b3315 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:41.450627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94059358-c1aa-4da7-ae25-52542903e031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:41.451572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94059358-c1aa-4da7-ae25-52542903e031 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.451847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94059358-c1aa-4da7-ae25-52542903e031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.452086 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94059358-c1aa-4da7-ae25-52542903e031 admin admin] PATH_INFO: `/v3/projects/8eba05f9e3a8402f96924c5c715229a5/users/a6346c2aa3ab4ebabd9bbd0b9c441ad3/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.467124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94059358-c1aa-4da7-ae25-52542903e031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eba05f9e3a8402f96924c5c715229a5, user_id=a6346c2aa3ab4ebabd9bbd0b9c441ad3, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:41.469799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94059358-c1aa-4da7-ae25-52542903e031 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:41.484207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1189/2384] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:37:41 2022] PUT /identity/v3/projects/8eba05f9e3a8402f96924c5c715229a5/users/a6346c2aa3ab4ebabd9bbd0b9c441ad3/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:37:41.489461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b16122-0315-467e-bd72-49235ed4117d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.489753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b16122-0315-467e-bd72-49235ed4117d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.490584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b16122-0315-467e-bd72-49235ed4117d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.508799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e6b16122-0315-467e-bd72-49235ed4117d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:37:41.509974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e6b16122-0315-467e-bd72-49235ed4117d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:41.588315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1196/2385] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:37:41.598175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94059358-c1aa-4da7-ae25-52542903e031 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:41.600298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-94059358-c1aa-4da7-ae25-52542903e031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:41.623851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:37:41.662508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94059358-c1aa-4da7-ae25-52542903e031 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:41.666553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d7331-2a11-4556-9d27-0fe8199cd0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:41.667229 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d7331-2a11-4556-9d27-0fe8199cd0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:41.667551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d7331-2a11-4556-9d27-0fe8199cd0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:41.667829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d7331-2a11-4556-9d27-0fe8199cd0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:41.677478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d7331-2a11-4556-9d27-0fe8199cd0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:41.679118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d7331-2a11-4556-9d27-0fe8199cd0a1 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:41.691198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1190/2386] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:37:42.318854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50c5ae-339d-41ff-bb5e-fba2e61e1801 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:42.319435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50c5ae-339d-41ff-bb5e-fba2e61e1801 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:42.319435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50c5ae-339d-41ff-bb5e-fba2e61e1801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:42.347125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fd50c5ae-339d-41ff-bb5e-fba2e61e1801 None None] MFA Rules not processed for user `a6346c2aa3ab4ebabd9bbd0b9c441ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:37:42.348391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fd50c5ae-339d-41ff-bb5e-fba2e61e1801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:42.365751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:37:42.424744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1197/2387] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:37:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:37:42.435897 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d7331-2a11-4556-9d27-0fe8199cd0a1 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:42.437375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-447d7331-2a11-4556-9d27-0fe8199cd0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:42.517897 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d7331-2a11-4556-9d27-0fe8199cd0a1 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:42.522348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a2980-0d35-4f63-b370-a3510fc3147a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:42.523008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a2980-0d35-4f63-b370-a3510fc3147a service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:42.523307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a2980-0d35-4f63-b370-a3510fc3147a service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:42.523592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a2980-0d35-4f63-b370-a3510fc3147a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:42.532378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a2980-0d35-4f63-b370-a3510fc3147a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:42.533827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a2980-0d35-4f63-b370-a3510fc3147a service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:42.544384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1191/2388] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:37:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:37:47.302229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81828201-737b-4c03-a372-1a9ad7032de5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:47.302751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81828201-737b-4c03-a372-1a9ad7032de5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:47.302867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81828201-737b-4c03-a372-1a9ad7032de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:47.363172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-81828201-737b-4c03-a372-1a9ad7032de5 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:37:47.365051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-81828201-737b-4c03-a372-1a9ad7032de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:47.444023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1198/2389] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:37:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:37:47.489163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a2980-0d35-4f63-b370-a3510fc3147a service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:37:47.490716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b44a2980-0d35-4f63-b370-a3510fc3147a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:47.586559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a2980-0d35-4f63-b370-a3510fc3147a service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:37:47.593407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3aff5-440a-42e5-b925-9b3434877dfa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:37:47.594049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3aff5-440a-42e5-b925-9b3434877dfa service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:47.594332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3aff5-440a-42e5-b925-9b3434877dfa service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:47.594608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3aff5-440a-42e5-b925-9b3434877dfa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:47.603593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3aff5-440a-42e5-b925-9b3434877dfa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:37:47.605217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3aff5-440a-42e5-b925-9b3434877dfa service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:37:47.613620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1192/2390] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:37:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:37:48.175570 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84175f-ab3d-4fc0-89e9-8b6e4e9b4eff None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:37:48.175906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84175f-ab3d-4fc0-89e9-8b6e4e9b4eff None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:37:48.176284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84175f-ab3d-4fc0-89e9-8b6e4e9b4eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:37:48.200284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6f84175f-ab3d-4fc0-89e9-8b6e4e9b4eff None None] MFA Rules not processed for user `a6346c2aa3ab4ebabd9bbd0b9c441ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:37:48.201538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6f84175f-ab3d-4fc0-89e9-8b6e4e9b4eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:37:48.209511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1199/2391] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:37:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:37:57.610418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:37:58.191060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=69 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:38:02.994799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595b61b-ab41-480a-952b-9a0b18ae5059 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:38:02.995547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595b61b-ab41-480a-952b-9a0b18ae5059 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:38:02.995547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595b61b-ab41-480a-952b-9a0b18ae5059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:38:03.013473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-4595b61b-ab41-480a-952b-9a0b18ae5059 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:38:03.014722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4595b61b-ab41-480a-952b-9a0b18ae5059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:38:03.022695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1193/2392] 10.222.0.42 () {58 vars in 967 bytes} [Sat Jul 2 03:38:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:38:03.036512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84175f-ab3d-4fc0-89e9-8b6e4e9b4eff None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:38:03.050008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84175f-ab3d-4fc0-89e9-8b6e4e9b4eff None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:38:03.056307 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35aa076-cac6-43ab-a205-6bd94588acd6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:38:03.056950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35aa076-cac6-43ab-a205-6bd94588acd6 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:38:03.057233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35aa076-cac6-43ab-a205-6bd94588acd6 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:38:03.057504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35aa076-cac6-43ab-a205-6bd94588acd6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:38:03.066354 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35aa076-cac6-43ab-a205-6bd94588acd6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:38:03.069171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35aa076-cac6-43ab-a205-6bd94588acd6 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:38:03.079272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1200/2393] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:38:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:38:05.143512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595b61b-ab41-480a-952b-9a0b18ae5059 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:38:05.145163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4595b61b-ab41-480a-952b-9a0b18ae5059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:38:05.161423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595b61b-ab41-480a-952b-9a0b18ae5059 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:38:05.167228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:38:05.168611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:38:05.168909 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:38:05.169529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] PATH_INFO: `/v3/users/662efed3aab046bfb378bbffc49a9c11` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:38:05.171180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=662efed3aab046bfb378bbffc49a9c11)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:38:05.173584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:38:05.227586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662efed3aab046bfb378bbffc49a9c11'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.228358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662efed3aab046bfb378bbffc49a9c11'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.236888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662efed3aab046bfb378bbffc49a9c11'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.237528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662efed3aab046bfb378bbffc49a9c11'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.251129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e48795ad-595a-4051-9377-e465502ec595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662efed3aab046bfb378bbffc49a9c11'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.265629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1194/2394] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:38:05 2022] DELETE /identity/v3/users/662efed3aab046bfb378bbffc49a9c11 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:38:05.455807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35aa076-cac6-43ab-a205-6bd94588acd6 service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:38:05.469956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35aa076-cac6-43ab-a205-6bd94588acd6 service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:38:05.537865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:38:05.538734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:38:05.539064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:38:05.539366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] PATH_INFO: `/v3/projects/81a1413099324b758a2608fa5d67c0a3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:38:05.540750 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a1413099324b758a2608fa5d67c0a3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:38:05.542983 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:38:05.602284 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a1413099324b758a2608fa5d67c0a3', 'request_id': 'req-ccef1115-f971-4c60-9e44-e05a59d0c21e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.610043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a1413099324b758a2608fa5d67c0a3', 'request_id': 'req-ccef1115-f971-4c60-9e44-e05a59d0c21e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.617587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a1413099324b758a2608fa5d67c0a3', 'request_id': 'req-ccef1115-f971-4c60-9e44-e05a59d0c21e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.638422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] The token cache is being invalidate because project 81a1413099324b758a2608fa5d67c0a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:38:05.638708 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.639904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ccef1115-f971-4c60-9e44-e05a59d0c21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:38:05.641073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1201/2395] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:38:05 2022] DELETE /identity/v3/projects/81a1413099324b758a2608fa5d67c0a3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:38:15.261799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:38:15.635486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:38:17.159536 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8772ea-ce33-46f7-a59b-79cb9e31e18c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:38:17.160085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8772ea-ce33-46f7-a59b-79cb9e31e18c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:38:17.160085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8772ea-ce33-46f7-a59b-79cb9e31e18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:38:17.178387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5d8772ea-ce33-46f7-a59b-79cb9e31e18c None None] MFA Rules not processed for user `868e2f0e822b430981369d414eee8e53`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:38:17.179628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d8772ea-ce33-46f7-a59b-79cb9e31e18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:38:17.241395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1195/2396] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:38:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:38:27.233830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=25 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:38:57.278296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e30057-d62a-4c9c-88ec-45fad256920a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:38:57.278796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e30057-d62a-4c9c-88ec-45fad256920a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:38:57.278894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e30057-d62a-4c9c-88ec-45fad256920a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:38:57.298259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-88e30057-d62a-4c9c-88ec-45fad256920a None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:38:57.299436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-88e30057-d62a-4c9c-88ec-45fad256920a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:38:57.364828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1202/2397] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:38:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:38:57.387323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8772ea-ce33-46f7-a59b-79cb9e31e18c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:38:57.388866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5d8772ea-ce33-46f7-a59b-79cb9e31e18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:38:57.460945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8772ea-ce33-46f7-a59b-79cb9e31e18c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:38:57.467141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f4f27-4f68-4d1c-ac81-e022357f8ed6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:38:57.467830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f4f27-4f68-4d1c-ac81-e022357f8ed6 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:38:57.468110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f4f27-4f68-4d1c-ac81-e022357f8ed6 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:38:57.468380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f4f27-4f68-4d1c-ac81-e022357f8ed6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:38:57.477531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f4f27-4f68-4d1c-ac81-e022357f8ed6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:38:57.479141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f4f27-4f68-4d1c-ac81-e022357f8ed6 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:38:57.487343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1196/2398] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:38:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:39:07.357193 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=25 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:39:07.483996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=32 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:39:18.532448 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f0c3-0ed1-4c7f-847a-1a89771feb50 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:18.532936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f0c3-0ed1-4c7f-847a-1a89771feb50 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:18.533054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4f0c3-0ed1-4c7f-847a-1a89771feb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:18.553920 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3ca4f0c3-0ed1-4c7f-847a-1a89771feb50 None None] MFA Rules not processed for user `a6346c2aa3ab4ebabd9bbd0b9c441ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:39:18.555536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3ca4f0c3-0ed1-4c7f-847a-1a89771feb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:18.614370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1203/2399] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:39:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:39:20.174575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f479ff-270b-4adc-965f-ea580dbb5392 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:20.175177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f479ff-270b-4adc-965f-ea580dbb5392 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:20.175177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f479ff-270b-4adc-965f-ea580dbb5392 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:20.176540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1197/2400] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:39:20.182946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e224aa-4e0e-4806-9f6d-2722ae0a2aa2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:20.183494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e224aa-4e0e-4806-9f6d-2722ae0a2aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:20.183494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e224aa-4e0e-4806-9f6d-2722ae0a2aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:20.239621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f1e224aa-4e0e-4806-9f6d-2722ae0a2aa2 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:39:20.245469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f1e224aa-4e0e-4806-9f6d-2722ae0a2aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:20.354597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1204/2401] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:39:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:39:20.366882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f479ff-270b-4adc-965f-ea580dbb5392 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:39:20.369003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f3f479ff-270b-4adc-965f-ea580dbb5392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:20.469078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f479ff-270b-4adc-965f-ea580dbb5392 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:39:20.473032 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec6465-8f5e-44a7-89cf-75fbfd7b6d5d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:39:20.473750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec6465-8f5e-44a7-89cf-75fbfd7b6d5d service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:20.474160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec6465-8f5e-44a7-89cf-75fbfd7b6d5d service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:20.474501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec6465-8f5e-44a7-89cf-75fbfd7b6d5d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:20.500468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec6465-8f5e-44a7-89cf-75fbfd7b6d5d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:39:20.501965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec6465-8f5e-44a7-89cf-75fbfd7b6d5d service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:39:20.514072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1198/2402] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:39:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:39:20.581152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101e1e9-1de0-4999-b78d-3ebf80e57d23 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:20.581442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101e1e9-1de0-4999-b78d-3ebf80e57d23 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:20.581765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101e1e9-1de0-4999-b78d-3ebf80e57d23 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:20.583176 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1205/2403] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:39:20.588342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053a963-88ab-4cd8-8478-951cbe9663cd None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:20.588796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053a963-88ab-4cd8-8478-951cbe9663cd None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:20.588990 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053a963-88ab-4cd8-8478-951cbe9663cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:20.608412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5053a963-88ab-4cd8-8478-951cbe9663cd None None] MFA Rules not processed for user `d67be26446324c91b956e9d930e04b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:39:20.609612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5053a963-88ab-4cd8-8478-951cbe9663cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:20.665316 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1199/2404] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:39:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:39:20.677370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101e1e9-1de0-4999-b78d-3ebf80e57d23 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:39:20.679172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7101e1e9-1de0-4999-b78d-3ebf80e57d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:20.693095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101e1e9-1de0-4999-b78d-3ebf80e57d23 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:39:20.697170 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9aeb8-e0a8-4873-a934-08e039d038f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:39:20.697968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9aeb8-e0a8-4873-a934-08e039d038f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:20.698248 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9aeb8-e0a8-4873-a934-08e039d038f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:20.698521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9aeb8-e0a8-4873-a934-08e039d038f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:20.707858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9aeb8-e0a8-4873-a934-08e039d038f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:39:20.709390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9aeb8-e0a8-4873-a934-08e039d038f3 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:39:20.718285 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1206/2405] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:39:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:39:20.803583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f1e38-1f1a-4762-b1ae-c6b66b1e2645 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:20.803839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f1e38-1f1a-4762-b1ae-c6b66b1e2645 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:20.804152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f1e38-1f1a-4762-b1ae-c6b66b1e2645 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:20.805768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1200/2406] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:39:20.832608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7acea-1ce2-4516-be9e-daf4f6e2733a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:20.832856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7acea-1ce2-4516-be9e-daf4f6e2733a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:20.833150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7acea-1ce2-4516-be9e-daf4f6e2733a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:20.857141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dfb7acea-1ce2-4516-be9e-daf4f6e2733a None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:39:20.858475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dfb7acea-1ce2-4516-be9e-daf4f6e2733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:20.870265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1207/2407] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:39:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:39:20.881149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f1e38-1f1a-4762-b1ae-c6b66b1e2645 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:39:20.892237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f1e38-1f1a-4762-b1ae-c6b66b1e2645 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:39:20.895808 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290f7be-9e96-4d63-8d20-98cfb840f709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:39:20.896458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290f7be-9e96-4d63-8d20-98cfb840f709 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:20.896744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290f7be-9e96-4d63-8d20-98cfb840f709 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:20.897023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290f7be-9e96-4d63-8d20-98cfb840f709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:20.908384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290f7be-9e96-4d63-8d20-98cfb840f709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:39:20.909832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290f7be-9e96-4d63-8d20-98cfb840f709 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:39:20.922079 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1201/2408] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:39:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:39:21.467444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7acea-1ce2-4516-be9e-daf4f6e2733a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:39:21.479180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7acea-1ce2-4516-be9e-daf4f6e2733a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:39:21.485036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52aa256-f4c6-49b4-ab6d-f88fe9dc48a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:39:21.486477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52aa256-f4c6-49b4-ab6d-f88fe9dc48a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:21.486915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52aa256-f4c6-49b4-ab6d-f88fe9dc48a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:21.487540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52aa256-f4c6-49b4-ab6d-f88fe9dc48a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:21.489924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e52aa256-f4c6-49b4-ab6d-f88fe9dc48a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:21.508527 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52aa256-f4c6-49b4-ab6d-f88fe9dc48a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:39:21.510092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52aa256-f4c6-49b4-ab6d-f88fe9dc48a1 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:39:21.524951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1208/2409] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:39:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:39:27.010233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90aeee4-67ff-4111-8c18-72cdbf024b1a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:27.010732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90aeee4-67ff-4111-8c18-72cdbf024b1a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:27.010826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90aeee4-67ff-4111-8c18-72cdbf024b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:27.028555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e90aeee4-67ff-4111-8c18-72cdbf024b1a None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:39:27.029765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e90aeee4-67ff-4111-8c18-72cdbf024b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:27.037083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1202/2410] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:39:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:39:27.049850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52aa256-f4c6-49b4-ab6d-f88fe9dc48a1 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:39:27.061490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52aa256-f4c6-49b4-ab6d-f88fe9dc48a1 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:39:27.066428 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec6894-fbe7-4621-85a1-01f5d9a689b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:39:27.067078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec6894-fbe7-4621-85a1-01f5d9a689b7 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:27.067387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec6894-fbe7-4621-85a1-01f5d9a689b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:27.067656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec6894-fbe7-4621-85a1-01f5d9a689b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:27.076135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec6894-fbe7-4621-85a1-01f5d9a689b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:39:27.077731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec6894-fbe7-4621-85a1-01f5d9a689b7 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:39:27.086210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1209/2411] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:39:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:39:27.548089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90aeee4-67ff-4111-8c18-72cdbf024b1a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:39:27.558608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90aeee4-67ff-4111-8c18-72cdbf024b1a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:39:27.562835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:39:27.563598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:27.564031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:27.564399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:27.566324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:27.581933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:39:27.583518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:39:27.595129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1203/2412] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:39:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:39:29.955853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec6894-fbe7-4621-85a1-01f5d9a689b7 service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:39:29.957701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-25ec6894-fbe7-4621-85a1-01f5d9a689b7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:29.975613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec6894-fbe7-4621-85a1-01f5d9a689b7 service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:39:29.983116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:39:29.983831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:29.984113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:29.984387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:29.986579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:30.002510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:39:30.060182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:39:30.062125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:39:30.072410 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1210/2413] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:39:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:39:37.590113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=73 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:39:40.067470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=20 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:39:47.081785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:39:47.083454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:47.094967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd2438-fd17-4aa3-b14a-a9026267f7e5 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:39:47.099791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72713d4d-9256-40c4-a8b3-b31933cd6fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:39:47.100410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72713d4d-9256-40c4-a8b3-b31933cd6fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:47.100714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72713d4d-9256-40c4-a8b3-b31933cd6fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:47.100989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72713d4d-9256-40c4-a8b3-b31933cd6fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:47.103087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-72713d4d-9256-40c4-a8b3-b31933cd6fb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:47.114425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72713d4d-9256-40c4-a8b3-b31933cd6fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:39:47.115956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72713d4d-9256-40c4-a8b3-b31933cd6fb4 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:39:47.124317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1204/2414] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:39:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:39:48.335121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:39:48.337844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:48.357291 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4622e-f319-4482-bb66-666af6cb1cc0 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:39:48.365087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f918f-389e-4168-8016-c3b44fbdc4e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:39:48.365667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f918f-389e-4168-8016-c3b44fbdc4e0 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:39:48.365938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f918f-389e-4168-8016-c3b44fbdc4e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:39:48.366212 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f918f-389e-4168-8016-c3b44fbdc4e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:39:48.368419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-be1f918f-389e-4168-8016-c3b44fbdc4e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:39:48.436968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f918f-389e-4168-8016-c3b44fbdc4e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:39:48.438568 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f918f-389e-4168-8016-c3b44fbdc4e0 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:39:48.448171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1211/2415] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:39:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:39:57.120738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:39:58.444023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=28 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:40:01.452977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4df93e-1b14-47ed-9fd4-ff2f3e57d577 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:01.453467 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4df93e-1b14-47ed-9fd4-ff2f3e57d577 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:01.453603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4df93e-1b14-47ed-9fd4-ff2f3e57d577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:01.471540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bc4df93e-1b14-47ed-9fd4-ff2f3e57d577 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:40:01.472738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bc4df93e-1b14-47ed-9fd4-ff2f3e57d577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:01.480259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1205/2416] 10.222.0.42 () {58 vars in 967 bytes} [Sat Jul 2 03:40:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:40:01.501907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f918f-389e-4168-8016-c3b44fbdc4e0 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:01.514600 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f918f-389e-4168-8016-c3b44fbdc4e0 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:01.521349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4e28c-2157-4acf-9dac-d1c8e3c029ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:01.521987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4e28c-2157-4acf-9dac-d1c8e3c029ef service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:01.522264 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4e28c-2157-4acf-9dac-d1c8e3c029ef service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:01.522534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4e28c-2157-4acf-9dac-d1c8e3c029ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:01.532019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4e28c-2157-4acf-9dac-d1c8e3c029ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:01.533623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4e28c-2157-4acf-9dac-d1c8e3c029ef service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:01.542220 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1212/2417] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:40:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:40:11.466224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:40:11.538691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:40:23.979472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386e8b5-0324-4bb6-b6df-59b145adf8f7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:23.980027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386e8b5-0324-4bb6-b6df-59b145adf8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:23.980119 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386e8b5-0324-4bb6-b6df-59b145adf8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:23.998639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1386e8b5-0324-4bb6-b6df-59b145adf8f7 None None] MFA Rules not processed for user `c941dc00e0584666aba13263b8c1fcb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:40:23.999994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1386e8b5-0324-4bb6-b6df-59b145adf8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:24.007382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1206/2418] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:40:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:40:27.537739 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4e28c-2157-4acf-9dac-d1c8e3c029ef service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:27.550523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4e28c-2157-4acf-9dac-d1c8e3c029ef service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:27.556459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf64a9-0bd1-4115-884d-8adc85ff8a53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:27.557113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf64a9-0bd1-4115-884d-8adc85ff8a53 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:27.557396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf64a9-0bd1-4115-884d-8adc85ff8a53 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:27.557673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf64a9-0bd1-4115-884d-8adc85ff8a53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:27.559511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-00bf64a9-0bd1-4115-884d-8adc85ff8a53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:27.573245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf64a9-0bd1-4115-884d-8adc85ff8a53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:27.574860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf64a9-0bd1-4115-884d-8adc85ff8a53 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:27.586761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1213/2419] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:40:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:40:32.687449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386e8b5-0324-4bb6-b6df-59b145adf8f7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:32.702961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386e8b5-0324-4bb6-b6df-59b145adf8f7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:32.713356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc62aaa-02f0-48e4-a705-ca53b6e3afb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:32.714118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc62aaa-02f0-48e4-a705-ca53b6e3afb2 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:32.714472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc62aaa-02f0-48e4-a705-ca53b6e3afb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:32.714814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc62aaa-02f0-48e4-a705-ca53b6e3afb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:32.716963 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8fc62aaa-02f0-48e4-a705-ca53b6e3afb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:32.729853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc62aaa-02f0-48e4-a705-ca53b6e3afb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:32.731483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc62aaa-02f0-48e4-a705-ca53b6e3afb2 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:32.742709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1207/2420] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:40:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:40:32.780881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf64a9-0bd1-4115-884d-8adc85ff8a53 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:32.792919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf64a9-0bd1-4115-884d-8adc85ff8a53 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:32.798668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052b10-0069-4fe5-8d72-2a9111d73b52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:32.799406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052b10-0069-4fe5-8d72-2a9111d73b52 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:32.799755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052b10-0069-4fe5-8d72-2a9111d73b52 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:32.800096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052b10-0069-4fe5-8d72-2a9111d73b52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:32.802013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-76052b10-0069-4fe5-8d72-2a9111d73b52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:32.862725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052b10-0069-4fe5-8d72-2a9111d73b52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:32.864497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76052b10-0069-4fe5-8d72-2a9111d73b52 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:32.874904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1214/2421] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:40:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:40:37.986827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc62aaa-02f0-48e4-a705-ca53b6e3afb2 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:38.007956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc62aaa-02f0-48e4-a705-ca53b6e3afb2 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:38.014861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965202b-bdbe-48a0-b319-58240ac08d70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:38.015989 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965202b-bdbe-48a0-b319-58240ac08d70 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:38.016419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965202b-bdbe-48a0-b319-58240ac08d70 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:38.016993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965202b-bdbe-48a0-b319-58240ac08d70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:38.019825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4965202b-bdbe-48a0-b319-58240ac08d70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:38.040384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965202b-bdbe-48a0-b319-58240ac08d70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:38.042613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965202b-bdbe-48a0-b319-58240ac08d70 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:38.057796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1208/2422] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:40:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:40:39.031764 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052b10-0069-4fe5-8d72-2a9111d73b52 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:39.041889 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052b10-0069-4fe5-8d72-2a9111d73b52 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:39.045931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85e69d-f08b-47f1-bfc3-d306883aef40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:39.046595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85e69d-f08b-47f1-bfc3-d306883aef40 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:39.046883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85e69d-f08b-47f1-bfc3-d306883aef40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:39.047143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85e69d-f08b-47f1-bfc3-d306883aef40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:39.048969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8c85e69d-f08b-47f1-bfc3-d306883aef40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:39.059086 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85e69d-f08b-47f1-bfc3-d306883aef40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:39.060671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85e69d-f08b-47f1-bfc3-d306883aef40 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:39.068969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1215/2423] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:40:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:40:39.180489 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5edbe-74c1-487a-afcd-e077047a4f63 None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:39.181158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5edbe-74c1-487a-afcd-e077047a4f63 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:39.181158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5edbe-74c1-487a-afcd-e077047a4f63 None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:39.182731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1209/2424] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:40:39.191146 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941766d2-b5c2-4e33-af37-e901d0197601 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:39.191473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941766d2-b5c2-4e33-af37-e901d0197601 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:39.192065 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941766d2-b5c2-4e33-af37-e901d0197601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:39.211956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-941766d2-b5c2-4e33-af37-e901d0197601 None None] MFA Rules not processed for user `d67be26446324c91b956e9d930e04b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:40:39.213202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-941766d2-b5c2-4e33-af37-e901d0197601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:39.220383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1216/2425] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:40:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:40:39.235452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5edbe-74c1-487a-afcd-e077047a4f63 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:39.248148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5edbe-74c1-487a-afcd-e077047a4f63 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:39.252776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3b81e-ff82-4bf3-afcf-5bf819f80827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:39.253498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3b81e-ff82-4bf3-afcf-5bf819f80827 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:39.253963 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3b81e-ff82-4bf3-afcf-5bf819f80827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:39.254246 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3b81e-ff82-4bf3-afcf-5bf819f80827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:39.262630 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3b81e-ff82-4bf3-afcf-5bf819f80827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:39.264155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3b81e-ff82-4bf3-afcf-5bf819f80827 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:39.272987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1210/2426] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:40:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:40:39.368181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaf71b-7e70-4aa5-a910-229418ce42c9 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:39.368474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaf71b-7e70-4aa5-a910-229418ce42c9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:39.368778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efaf71b-7e70-4aa5-a910-229418ce42c9 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:39.370198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1217/2427] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:40:39.375358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69b724-58c3-49ac-a6a4-cebd22886642 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:39.375650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69b724-58c3-49ac-a6a4-cebd22886642 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:39.375955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69b724-58c3-49ac-a6a4-cebd22886642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:39.394697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-3a69b724-58c3-49ac-a6a4-cebd22886642 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:40:39.396011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a69b724-58c3-49ac-a6a4-cebd22886642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:39.405983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1211/2428] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:40:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:40:39.416506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efaf71b-7e70-4aa5-a910-229418ce42c9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:39.430798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efaf71b-7e70-4aa5-a910-229418ce42c9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:39.436203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aba484-80e3-459a-b44c-1da1a6d6be89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:39.436833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aba484-80e3-459a-b44c-1da1a6d6be89 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:39.437116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aba484-80e3-459a-b44c-1da1a6d6be89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:39.437392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aba484-80e3-459a-b44c-1da1a6d6be89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:39.449081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aba484-80e3-459a-b44c-1da1a6d6be89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:39.450534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aba484-80e3-459a-b44c-1da1a6d6be89 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:39.461847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1218/2429] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:40:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:40:39.532134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69b724-58c3-49ac-a6a4-cebd22886642 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:39.534722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3a69b724-58c3-49ac-a6a4-cebd22886642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:39.618656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69b724-58c3-49ac-a6a4-cebd22886642 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:39.625085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:39.625913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:39.626198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:39.626535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:39.628832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:39.641750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:39.643568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:39.654192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1212/2430] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:40:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:40:49.457056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=49 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:40:49.650169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=56 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:40:59.020062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aba484-80e3-459a-b44c-1da1a6d6be89 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:59.031578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aba484-80e3-459a-b44c-1da1a6d6be89 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:59.036968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89d7c3-ae64-48ca-8dda-3303bc4a3d89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:59.037710 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89d7c3-ae64-48ca-8dda-3303bc4a3d89 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:59.038070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89d7c3-ae64-48ca-8dda-3303bc4a3d89 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:59.038412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89d7c3-ae64-48ca-8dda-3303bc4a3d89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:59.040369 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cd89d7c3-ae64-48ca-8dda-3303bc4a3d89 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:59.053209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89d7c3-ae64-48ca-8dda-3303bc4a3d89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:59.054848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89d7c3-ae64-48ca-8dda-3303bc4a3d89 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:59.065347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1219/2431] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:40:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:40:59.082259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:59.083755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:40:59.154991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f24415-b7e7-4d6c-8ba7-7a8d3b9e61a5 service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:59.159525 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:59.160433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:59.160714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:59.160990 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] PATH_INFO: `/v3/users/a6346c2aa3ab4ebabd9bbd0b9c441ad3` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:59.162265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6346c2aa3ab4ebabd9bbd0b9c441ad3)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:59.164266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:59.221763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6346c2aa3ab4ebabd9bbd0b9c441ad3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.222473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6346c2aa3ab4ebabd9bbd0b9c441ad3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.231479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6346c2aa3ab4ebabd9bbd0b9c441ad3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.232210 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6346c2aa3ab4ebabd9bbd0b9c441ad3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.245116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6346c2aa3ab4ebabd9bbd0b9c441ad3'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.257868 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1213/2432] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:40:59 2022] DELETE /identity/v3/users/a6346c2aa3ab4ebabd9bbd0b9c441ad3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:40:59.445432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89d7c3-ae64-48ca-8dda-3303bc4a3d89 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:40:59.456818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89d7c3-ae64-48ca-8dda-3303bc4a3d89 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:40:59.518415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:40:59.519279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:40:59.519586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:40:59.519860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] PATH_INFO: `/v3/projects/8eba05f9e3a8402f96924c5c715229a5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:40:59.521019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eba05f9e3a8402f96924c5c715229a5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:40:59.522943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:40:59.570173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eba05f9e3a8402f96924c5c715229a5', 'request_id': 'req-8fbe63e1-af78-42b0-8980-4dfa2604fa22'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.576148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eba05f9e3a8402f96924c5c715229a5', 'request_id': 'req-8fbe63e1-af78-42b0-8980-4dfa2604fa22'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.582189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eba05f9e3a8402f96924c5c715229a5', 'request_id': 'req-8fbe63e1-af78-42b0-8980-4dfa2604fa22'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.598787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] The token cache is being invalidate because project 8eba05f9e3a8402f96924c5c715229a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:40:59.599144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.600159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fbe63e1-af78-42b0-8980-4dfa2604fa22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:40:59.602681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1220/2433] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:40:59 2022] DELETE /identity/v3/projects/8eba05f9e3a8402f96924c5c715229a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:41:05.213870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:41:05.216431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:41:05.299958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a5747-8e93-43bc-a625-9620e9544770 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:41:05.313469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:41:05.373902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:41:05.374561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:41:05.374842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:41:05.375110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:41:05.377141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:41:05.437414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:41:05.439017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:41:05.448645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1214/2434] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:41:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:41:09.593946 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:41:15.444095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=48 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:41:17.093526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbb87b-c043-468a-bc77-9affccddc67a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:41:17.094038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbb87b-c043-468a-bc77-9affccddc67a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:41:17.094154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbb87b-c043-468a-bc77-9affccddc67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:41:17.115452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-59bbb87b-c043-468a-bc77-9affccddc67a None None] MFA Rules not processed for user `868e2f0e822b430981369d414eee8e53`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:41:17.116722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-59bbb87b-c043-468a-bc77-9affccddc67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:41:17.125222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1221/2435] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:41:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:41:17.437819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:41:17.439461 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:41:17.498204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e55595-3e85-4b2e-a9f5-df0059c26960 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:41:17.501887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3895db-c3f1-41c3-9bfb-a42b8a39c1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:41:17.502531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3895db-c3f1-41c3-9bfb-a42b8a39c1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:41:17.502814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3895db-c3f1-41c3-9bfb-a42b8a39c1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:41:17.503090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3895db-c3f1-41c3-9bfb-a42b8a39c1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:41:17.505064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7f3895db-c3f1-41c3-9bfb-a42b8a39c1ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:41:17.514478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3895db-c3f1-41c3-9bfb-a42b8a39c1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:41:17.515987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3895db-c3f1-41c3-9bfb-a42b8a39c1ee service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:41:17.523958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1215/2436] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:41:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:41:27.109921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:41:27.520416 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=26 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:41:34.273076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e1c72-b9a3-4659-a703-65fd6d58c44c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:41:34.273604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e1c72-b9a3-4659-a703-65fd6d58c44c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:41:34.273709 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e1c72-b9a3-4659-a703-65fd6d58c44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:41:34.291342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-362e1c72-b9a3-4659-a703-65fd6d58c44c None None] MFA Rules not processed for user `112d731047a64efc8e9143c4d471a7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:41:34.292910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-362e1c72-b9a3-4659-a703-65fd6d58c44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:41:34.348148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1222/2437] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:41:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:41:44.341046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=25 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:41:57.908277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3895db-c3f1-41c3-9bfb-a42b8a39c1ee service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:41:57.918787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3895db-c3f1-41c3-9bfb-a42b8a39c1ee service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:41:57.924135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:41:57.924293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:41:57.924538 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:41:57.924824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:41:57.926874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:41:57.996129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:41:57.997652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:41:58.007895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1216/2438] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:01.368858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e1c72-b9a3-4659-a703-65fd6d58c44c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:01.370495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-362e1c72-b9a3-4659-a703-65fd6d58c44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:01.441622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:01.443397 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:01.456504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e1c72-b9a3-4659-a703-65fd6d58c44c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:01.457725 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3660fd77-596e-406d-a804-bf9991f32e74 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:01.462391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f891c6-09ca-4627-80c1-797ee2a0906d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:01.462613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:01.463314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f891c6-09ca-4627-80c1-797ee2a0906d service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:01.463526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:01.463774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f891c6-09ca-4627-80c1-797ee2a0906d service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:01.464061 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f891c6-09ca-4627-80c1-797ee2a0906d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:01.464539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:01.464824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] PATH_INFO: `/v3/users/c941dc00e0584666aba13263b8c1fcb8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:01.466107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c941dc00e0584666aba13263b8c1fcb8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:01.466410 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-50f891c6-09ca-4627-80c1-797ee2a0906d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:01.468140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:01.479827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f891c6-09ca-4627-80c1-797ee2a0906d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:01.481628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f891c6-09ca-4627-80c1-797ee2a0906d service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:01.492919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1223/2439] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:01.534943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c941dc00e0584666aba13263b8c1fcb8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.535958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c941dc00e0584666aba13263b8c1fcb8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.545425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c941dc00e0584666aba13263b8c1fcb8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.546284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c941dc00e0584666aba13263b8c1fcb8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.558664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-ebe04b03-a5ac-48ff-add9-9ff7957d1080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c941dc00e0584666aba13263b8c1fcb8'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.571618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1217/2440] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:01 2022] DELETE /identity/v3/users/c941dc00e0584666aba13263b8c1fcb8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:01.755469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f891c6-09ca-4627-80c1-797ee2a0906d service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:01.772629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f891c6-09ca-4627-80c1-797ee2a0906d service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:01.831956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:01.832827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:01.833108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:01.833383 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] PATH_INFO: `/v3/projects/53c88ba2f03f47538ca0eaeb08ccfc07` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:01.834669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c88ba2f03f47538ca0eaeb08ccfc07)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:01.836935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:01.886050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c88ba2f03f47538ca0eaeb08ccfc07', 'request_id': 'req-57e0f3ec-2892-4358-918c-efe07e0cba39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.891365 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c88ba2f03f47538ca0eaeb08ccfc07', 'request_id': 'req-57e0f3ec-2892-4358-918c-efe07e0cba39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.897584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c88ba2f03f47538ca0eaeb08ccfc07', 'request_id': 'req-57e0f3ec-2892-4358-918c-efe07e0cba39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.912580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] The token cache is being invalidate because project 53c88ba2f03f47538ca0eaeb08ccfc07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:01.912862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.913472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-57e0f3ec-2892-4358-918c-efe07e0cba39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:01.915241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1224/2441] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:01 2022] DELETE /identity/v3/projects/53c88ba2f03f47538ca0eaeb08ccfc07 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:01.920705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f029d40-c064-47f8-af36-9482c4286192 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:01.920996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f029d40-c064-47f8-af36-9482c4286192 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:01.921276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f029d40-c064-47f8-af36-9482c4286192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:01.940404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6f029d40-c064-47f8-af36-9482c4286192 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:01.941701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6f029d40-c064-47f8-af36-9482c4286192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:02.005634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1218/2442] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:42:02.014265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931cced-3146-48b0-82c0-47b684d04e32 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:02.014582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931cced-3146-48b0-82c0-47b684d04e32 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:02.014858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931cced-3146-48b0-82c0-47b684d04e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:02.034677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e931cced-3146-48b0-82c0-47b684d04e32 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:02.035919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e931cced-3146-48b0-82c0-47b684d04e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:02.045614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1225/2443] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:02.050224 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f029d40-c064-47f8-af36-9482c4286192 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:02.060036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f029d40-c064-47f8-af36-9482c4286192 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:02.064932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50d6e2-5b66-4a67-9f0d-5a2112f1407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:02.065625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50d6e2-5b66-4a67-9f0d-5a2112f1407d admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:02.065888 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50d6e2-5b66-4a67-9f0d-5a2112f1407d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:02.066155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec50d6e2-5b66-4a67-9f0d-5a2112f1407d admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:02.066885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50d6e2-5b66-4a67-9f0d-5a2112f1407d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:42:02.067192 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50d6e2-5b66-4a67-9f0d-5a2112f1407d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:02.068823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec50d6e2-5b66-4a67-9f0d-5a2112f1407d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:02.070304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:02.081213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1219/2444] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:42:02.085734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931cced-3146-48b0-82c0-47b684d04e32 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:02.095317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931cced-3146-48b0-82c0-47b684d04e32 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:02.100727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0c06b-df14-44c4-96dd-316a894dbb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:02.101387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0c06b-df14-44c4-96dd-316a894dbb70 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:02.101671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0c06b-df14-44c4-96dd-316a894dbb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:02.102040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0c06b-df14-44c4-96dd-316a894dbb70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:02.102770 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0c06b-df14-44c4-96dd-316a894dbb70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:02.104741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0c06b-df14-44c4-96dd-316a894dbb70 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:02.114970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:02.146208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1226/2445] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:02 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:02.150723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50d6e2-5b66-4a67-9f0d-5a2112f1407d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:02.161022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec50d6e2-5b66-4a67-9f0d-5a2112f1407d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:02.219514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c71735-f2af-4114-926b-e39025217324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:02.220180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c71735-f2af-4114-926b-e39025217324 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:02.220469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c71735-f2af-4114-926b-e39025217324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:02.220744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c71735-f2af-4114-926b-e39025217324 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:02.221429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c71735-f2af-4114-926b-e39025217324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:02.223369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c71735-f2af-4114-926b-e39025217324 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:02.245644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1220/2446] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:02 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:02.249790 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0c06b-df14-44c4-96dd-316a894dbb70 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:02.260884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0c06b-df14-44c4-96dd-316a894dbb70 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:02.265891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611d20a-75c5-4b83-b18d-131c368f6425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:02.266582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611d20a-75c5-4b83-b18d-131c368f6425 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:02.266864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611d20a-75c5-4b83-b18d-131c368f6425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:02.267139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611d20a-75c5-4b83-b18d-131c368f6425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:02.267867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611d20a-75c5-4b83-b18d-131c368f6425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:02.269514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611d20a-75c5-4b83-b18d-131c368f6425 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:02.279408 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1227/2447] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:02.283497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c71735-f2af-4114-926b-e39025217324 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:02.294026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c71735-f2af-4114-926b-e39025217324 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:02.298758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d61b3-ef57-48e2-8395-9f5422c2c5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:02.299447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d61b3-ef57-48e2-8395-9f5422c2c5fe admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:02.299731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d61b3-ef57-48e2-8395-9f5422c2c5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:02.300010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d61b3-ef57-48e2-8395-9f5422c2c5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:02.300728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d61b3-ef57-48e2-8395-9f5422c2c5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:02.302307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277d61b3-ef57-48e2-8395-9f5422c2c5fe admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:02.312096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1221/2448] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:02.316286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611d20a-75c5-4b83-b18d-131c368f6425 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:02.326685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611d20a-75c5-4b83-b18d-131c368f6425 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:02.331805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6625c8-6bb4-462f-ae45-40a7f3dcdf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:02.332949 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6625c8-6bb4-462f-ae45-40a7f3dcdf95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:02.333234 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6625c8-6bb4-462f-ae45-40a7f3dcdf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:02.333517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6625c8-6bb4-462f-ae45-40a7f3dcdf95 admin admin] PATH_INFO: `/v3/projects/4518933e0bef49e488cce3869465605b/users/759d856e88164f7089370be2af314efe/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:02.349370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6625c8-6bb4-462f-ae45-40a7f3dcdf95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4518933e0bef49e488cce3869465605b, user_id=759d856e88164f7089370be2af314efe, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:02.352215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6625c8-6bb4-462f-ae45-40a7f3dcdf95 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:02.366747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1228/2449] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:02 2022] PUT /identity/v3/projects/4518933e0bef49e488cce3869465605b/users/759d856e88164f7089370be2af314efe/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:42:02.377459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447744f-b839-4358-9c7f-91bb8af1da5f None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:02.377744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447744f-b839-4358-9c7f-91bb8af1da5f None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:02.378019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447744f-b839-4358-9c7f-91bb8af1da5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:02.405353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d447744f-b839-4358-9c7f-91bb8af1da5f None None] MFA Rules not processed for user `759d856e88164f7089370be2af314efe`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:02.406568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d447744f-b839-4358-9c7f-91bb8af1da5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:02.474240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1222/2450] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:02.488420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6625c8-6bb4-462f-ae45-40a7f3dcdf95 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:02.490597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9b6625c8-6bb4-462f-ae45-40a7f3dcdf95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:02.578325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6625c8-6bb4-462f-ae45-40a7f3dcdf95 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:02.583847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f9c4e-9c9b-4d61-a2e0-d5db9b10fe0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:02.584507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f9c4e-9c9b-4d61-a2e0-d5db9b10fe0d service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:02.584786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f9c4e-9c9b-4d61-a2e0-d5db9b10fe0d service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:02.585055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f9c4e-9c9b-4d61-a2e0-d5db9b10fe0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:02.593399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f9c4e-9c9b-4d61-a2e0-d5db9b10fe0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:02.595009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f9c4e-9c9b-4d61-a2e0-d5db9b10fe0d service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:02.604313 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1229/2451] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:11.017913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447744f-b839-4358-9c7f-91bb8af1da5f None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:11.081955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447744f-b839-4358-9c7f-91bb8af1da5f None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:11.087866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:11.088731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:11.089010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:11.089282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] PATH_INFO: `/v3/users/759d856e88164f7089370be2af314efe` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:11.090555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=759d856e88164f7089370be2af314efe)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:11.092504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:11.094828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:11.145138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '759d856e88164f7089370be2af314efe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.151165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '759d856e88164f7089370be2af314efe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.161986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '759d856e88164f7089370be2af314efe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.162675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '759d856e88164f7089370be2af314efe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.177325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '759d856e88164f7089370be2af314efe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.190256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1223/2452] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:11 2022] DELETE /identity/v3/users/759d856e88164f7089370be2af314efe => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:11.195052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63299c-33a0-408f-955e-2b28e9d60c8d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:11.195558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63299c-33a0-408f-955e-2b28e9d60c8d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:11.195673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63299c-33a0-408f-955e-2b28e9d60c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:11.214474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6a63299c-33a0-408f-955e-2b28e9d60c8d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:11.215738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6a63299c-33a0-408f-955e-2b28e9d60c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:11.291089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1230/2453] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:11.301365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:11.302853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:11.362454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad6ad7-eec3-4ba6-bf13-2820a2a44bc0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:11.367188 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020eff71-6120-433f-be09-240a17a90fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:11.367870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eff71-6120-433f-be09-240a17a90fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:11.368520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eff71-6120-433f-be09-240a17a90fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:11.368520 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eff71-6120-433f-be09-240a17a90fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:11.378230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020eff71-6120-433f-be09-240a17a90fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:11.379799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020eff71-6120-433f-be09-240a17a90fb6 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:11.389687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1224/2454] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:11.686547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63299c-33a0-408f-955e-2b28e9d60c8d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:11.688073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6a63299c-33a0-408f-955e-2b28e9d60c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:11.702953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63299c-33a0-408f-955e-2b28e9d60c8d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:11.707861 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:11.708890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:11.708985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:11.709342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] PATH_INFO: `/v3/projects/4518933e0bef49e488cce3869465605b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:11.710558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4518933e0bef49e488cce3869465605b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:11.712767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:11.748837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:11.765723 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4518933e0bef49e488cce3869465605b', 'request_id': 'req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.771010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4518933e0bef49e488cce3869465605b', 'request_id': 'req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.779302 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4518933e0bef49e488cce3869465605b', 'request_id': 'req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.797195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] The token cache is being invalidate because project 4518933e0bef49e488cce3869465605b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:11.797619 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.798443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d9eb8a32-b3bb-462c-8b30-a8456f40b6d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:11.800344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1231/2455] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:11 2022] DELETE /identity/v3/projects/4518933e0bef49e488cce3869465605b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:11.805653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010c486-4e09-453d-b626-0a63080d0dd4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:11.805940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010c486-4e09-453d-b626-0a63080d0dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:11.806216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010c486-4e09-453d-b626-0a63080d0dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:11.825926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6010c486-4e09-453d-b626-0a63080d0dd4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:11.827140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6010c486-4e09-453d-b626-0a63080d0dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:11.904456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1225/2456] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:11.912656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae0620-93c6-48d9-9e0b-4bf2bb596b53 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:11.912943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae0620-93c6-48d9-9e0b-4bf2bb596b53 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:11.913216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae0620-93c6-48d9-9e0b-4bf2bb596b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:11.934052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-39ae0620-93c6-48d9-9e0b-4bf2bb596b53 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:11.935169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-39ae0620-93c6-48d9-9e0b-4bf2bb596b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:11.943501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1232/2457] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:11.948311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010c486-4e09-453d-b626-0a63080d0dd4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:11.958864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010c486-4e09-453d-b626-0a63080d0dd4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:11.966153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cb7da-2fa0-4207-8140-a340ff195070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:11.966823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cb7da-2fa0-4207-8140-a340ff195070 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:11.967100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cb7da-2fa0-4207-8140-a340ff195070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:11.967401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cb7da-2fa0-4207-8140-a340ff195070 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:11.968128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4cb7da-2fa0-4207-8140-a340ff195070 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:42:11.968432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4cb7da-2fa0-4207-8140-a340ff195070 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:11.970046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4cb7da-2fa0-4207-8140-a340ff195070 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:11.982711 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1226/2458] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:42:11.987326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae0620-93c6-48d9-9e0b-4bf2bb596b53 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:11.997306 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae0620-93c6-48d9-9e0b-4bf2bb596b53 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:12.002827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c73c89-2271-461f-b4d5-0e18de11d8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:12.003558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c73c89-2271-461f-b4d5-0e18de11d8bf admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:12.003842 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c73c89-2271-461f-b4d5-0e18de11d8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:12.004143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c73c89-2271-461f-b4d5-0e18de11d8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:12.004838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c73c89-2271-461f-b4d5-0e18de11d8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:12.007111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c73c89-2271-461f-b4d5-0e18de11d8bf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:12.036610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1233/2459] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:11 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:12.041103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cb7da-2fa0-4207-8140-a340ff195070 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:12.055609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cb7da-2fa0-4207-8140-a340ff195070 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:12.064531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:12.117550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91678a71-f586-4a8c-9604-0a593e5fe22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:12.118227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91678a71-f586-4a8c-9604-0a593e5fe22e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:12.118707 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91678a71-f586-4a8c-9604-0a593e5fe22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:12.118846 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91678a71-f586-4a8c-9604-0a593e5fe22e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:12.119532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91678a71-f586-4a8c-9604-0a593e5fe22e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:12.121459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91678a71-f586-4a8c-9604-0a593e5fe22e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:12.143836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1227/2460] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:12 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:42:12.148133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c73c89-2271-461f-b4d5-0e18de11d8bf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:12.159013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c73c89-2271-461f-b4d5-0e18de11d8bf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:12.163855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bf82a-92fa-4d8c-853c-71258bcad6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:12.164518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bf82a-92fa-4d8c-853c-71258bcad6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:12.164823 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bf82a-92fa-4d8c-853c-71258bcad6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:12.165346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bf82a-92fa-4d8c-853c-71258bcad6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:12.166084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bf82a-92fa-4d8c-853c-71258bcad6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:12.168095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bf82a-92fa-4d8c-853c-71258bcad6d3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:12.178863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1234/2461] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:12.183160 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91678a71-f586-4a8c-9604-0a593e5fe22e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:12.193930 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91678a71-f586-4a8c-9604-0a593e5fe22e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:12.200249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ba506-69f4-41d6-acf7-07f08e2be8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:12.200911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ba506-69f4-41d6-acf7-07f08e2be8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:12.201191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ba506-69f4-41d6-acf7-07f08e2be8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:12.201447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ba506-69f4-41d6-acf7-07f08e2be8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:12.202164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ba506-69f4-41d6-acf7-07f08e2be8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:12.203807 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ba506-69f4-41d6-acf7-07f08e2be8b5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:12.214556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1228/2462] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:12.218864 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bf82a-92fa-4d8c-853c-71258bcad6d3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:12.229735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bf82a-92fa-4d8c-853c-71258bcad6d3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:12.234919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95516490-dc10-4497-9a52-f51a6c8a1a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:12.235850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95516490-dc10-4497-9a52-f51a6c8a1a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:12.236131 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95516490-dc10-4497-9a52-f51a6c8a1a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:12.236406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95516490-dc10-4497-9a52-f51a6c8a1a2a admin admin] PATH_INFO: `/v3/projects/1248a756a90748f8a6a75053c8cbdb6a/users/bf7b0f5196f1402185c6cf1f2b23f216/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:12.252534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95516490-dc10-4497-9a52-f51a6c8a1a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1248a756a90748f8a6a75053c8cbdb6a, user_id=bf7b0f5196f1402185c6cf1f2b23f216, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:12.255214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95516490-dc10-4497-9a52-f51a6c8a1a2a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:12.270306 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1235/2463] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:12 2022] PUT /identity/v3/projects/1248a756a90748f8a6a75053c8cbdb6a/users/bf7b0f5196f1402185c6cf1f2b23f216/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:12.282056 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06995480-8c2a-4d94-b0d4-dbb91e301d56 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:12.282344 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06995480-8c2a-4d94-b0d4-dbb91e301d56 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:12.282622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06995480-8c2a-4d94-b0d4-dbb91e301d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:12.311138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-06995480-8c2a-4d94-b0d4-dbb91e301d56 None None] MFA Rules not processed for user `bf7b0f5196f1402185c6cf1f2b23f216`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:12.312374 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-06995480-8c2a-4d94-b0d4-dbb91e301d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:12.382908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1229/2464] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:12.395318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95516490-dc10-4497-9a52-f51a6c8a1a2a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:12.398004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-95516490-dc10-4497-9a52-f51a6c8a1a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:12.481741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95516490-dc10-4497-9a52-f51a6c8a1a2a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:12.488960 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b2b84-3b0b-4696-8707-70136abd1958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:12.489617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b2b84-3b0b-4696-8707-70136abd1958 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:12.489903 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b2b84-3b0b-4696-8707-70136abd1958 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:12.490189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b2b84-3b0b-4696-8707-70136abd1958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:12.498368 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b2b84-3b0b-4696-8707-70136abd1958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:12.500009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b2b84-3b0b-4696-8707-70136abd1958 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:12.508880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1236/2465] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:15.239477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06995480-8c2a-4d94-b0d4-dbb91e301d56 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:15.305505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06995480-8c2a-4d94-b0d4-dbb91e301d56 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:15.310116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:15.311000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:15.311276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:15.311582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] PATH_INFO: `/v3/users/bf7b0f5196f1402185c6cf1f2b23f216` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:15.312924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf7b0f5196f1402185c6cf1f2b23f216)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:15.314877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:15.339264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:15.368075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf7b0f5196f1402185c6cf1f2b23f216'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.368729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf7b0f5196f1402185c6cf1f2b23f216'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.377987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf7b0f5196f1402185c6cf1f2b23f216'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.378721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf7b0f5196f1402185c6cf1f2b23f216'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.392381 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf7b0f5196f1402185c6cf1f2b23f216'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.406697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1230/2466] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:15 2022] DELETE /identity/v3/users/bf7b0f5196f1402185c6cf1f2b23f216 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:15.411559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249fbd4-3906-4925-98f4-4deddbd09181 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:15.412057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249fbd4-3906-4925-98f4-4deddbd09181 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:15.412161 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249fbd4-3906-4925-98f4-4deddbd09181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:15.430579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b249fbd4-3906-4925-98f4-4deddbd09181 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:15.431923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b249fbd4-3906-4925-98f4-4deddbd09181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:15.465858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:15.497420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1237/2467] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:15.507101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:15.508653 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:15.566903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38044b3-d6eb-45ad-a8f9-6ebc7b1bb92a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:15.570648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7c6d5-a268-4c8d-b6da-4b02f3a17e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:15.571760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7c6d5-a268-4c8d-b6da-4b02f3a17e2a service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:15.572305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7c6d5-a268-4c8d-b6da-4b02f3a17e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:15.572771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7c6d5-a268-4c8d-b6da-4b02f3a17e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:15.584760 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7c6d5-a268-4c8d-b6da-4b02f3a17e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:15.586218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7c6d5-a268-4c8d-b6da-4b02f3a17e2a service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:15.595954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1231/2468] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:15.877467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249fbd4-3906-4925-98f4-4deddbd09181 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:15.879005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b249fbd4-3906-4925-98f4-4deddbd09181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:15.892050 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249fbd4-3906-4925-98f4-4deddbd09181 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:15.897141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:15.898091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:15.898493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:15.898761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] PATH_INFO: `/v3/projects/1248a756a90748f8a6a75053c8cbdb6a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:15.900020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=1248a756a90748f8a6a75053c8cbdb6a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:15.902007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:15.953017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1248a756a90748f8a6a75053c8cbdb6a', 'request_id': 'req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.959644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1248a756a90748f8a6a75053c8cbdb6a', 'request_id': 'req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.966891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1248a756a90748f8a6a75053c8cbdb6a', 'request_id': 'req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.982135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] The token cache is being invalidate because project 1248a756a90748f8a6a75053c8cbdb6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:15.982414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.983043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ab7b3790-9828-4e74-bc1e-49a0a35d3dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:15.984651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1238/2469] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:15 2022] DELETE /identity/v3/projects/1248a756a90748f8a6a75053c8cbdb6a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:15.992062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304c5d5-2b8b-4aa5-84b2-65b5b209ced2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:15.992323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304c5d5-2b8b-4aa5-84b2-65b5b209ced2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:15.992716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304c5d5-2b8b-4aa5-84b2-65b5b209ced2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.012717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8304c5d5-2b8b-4aa5-84b2-65b5b209ced2 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:16.013958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8304c5d5-2b8b-4aa5-84b2-65b5b209ced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:16.073601 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1232/2470] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:16.082656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08926409-0b5c-42bc-b9bc-5c06dcb13d42 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.083015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08926409-0b5c-42bc-b9bc-5c06dcb13d42 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.083426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08926409-0b5c-42bc-b9bc-5c06dcb13d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.106873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-08926409-0b5c-42bc-b9bc-5c06dcb13d42 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:16.108116 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08926409-0b5c-42bc-b9bc-5c06dcb13d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:16.117380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1239/2471] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:16.122280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304c5d5-2b8b-4aa5-84b2-65b5b209ced2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.132632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304c5d5-2b8b-4aa5-84b2-65b5b209ced2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.138437 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd939-0ee0-4910-a146-ce4a97af94fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.139104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd939-0ee0-4910-a146-ce4a97af94fc admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.139426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd939-0ee0-4910-a146-ce4a97af94fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.139704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd939-0ee0-4910-a146-ce4a97af94fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.140452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bd939-0ee0-4910-a146-ce4a97af94fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:42:16.140736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bd939-0ee0-4910-a146-ce4a97af94fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.142354 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bd939-0ee0-4910-a146-ce4a97af94fc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.155455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1233/2472] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:42:16.161815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08926409-0b5c-42bc-b9bc-5c06dcb13d42 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.175890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08926409-0b5c-42bc-b9bc-5c06dcb13d42 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.184784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695cbc0-fd6b-4a1d-a307-c5ea6e831676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.185746 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695cbc0-fd6b-4a1d-a307-c5ea6e831676 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.186046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695cbc0-fd6b-4a1d-a307-c5ea6e831676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.186293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695cbc0-fd6b-4a1d-a307-c5ea6e831676 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.186970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695cbc0-fd6b-4a1d-a307-c5ea6e831676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.189860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695cbc0-fd6b-4a1d-a307-c5ea6e831676 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.220346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1240/2473] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:16 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:16.224665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd939-0ee0-4910-a146-ce4a97af94fc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.235698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd939-0ee0-4910-a146-ce4a97af94fc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.263414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:16.301873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f625e2d-ea4d-43d1-9837-67d5c0f6d2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.302550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f625e2d-ea4d-43d1-9837-67d5c0f6d2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.302836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f625e2d-ea4d-43d1-9837-67d5c0f6d2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.303114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f625e2d-ea4d-43d1-9837-67d5c0f6d2e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.303841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f625e2d-ea4d-43d1-9837-67d5c0f6d2e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.305799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f625e2d-ea4d-43d1-9837-67d5c0f6d2e9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.328455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1234/2474] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:16 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:16.332720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695cbc0-fd6b-4a1d-a307-c5ea6e831676 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.343937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695cbc0-fd6b-4a1d-a307-c5ea6e831676 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.348875 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46aec-0d29-432c-a30f-ea2758ebd723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.349532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46aec-0d29-432c-a30f-ea2758ebd723 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.349816 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46aec-0d29-432c-a30f-ea2758ebd723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.350098 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46aec-0d29-432c-a30f-ea2758ebd723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.350879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b46aec-0d29-432c-a30f-ea2758ebd723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.352583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b46aec-0d29-432c-a30f-ea2758ebd723 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.363616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1241/2475] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:16.367756 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f625e2d-ea4d-43d1-9837-67d5c0f6d2e9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.378638 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f625e2d-ea4d-43d1-9837-67d5c0f6d2e9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.383629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d96b76-e7f5-41e7-93a5-30a89892c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.384287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d96b76-e7f5-41e7-93a5-30a89892c174 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.384541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d96b76-e7f5-41e7-93a5-30a89892c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.384851 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d96b76-e7f5-41e7-93a5-30a89892c174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.386076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d96b76-e7f5-41e7-93a5-30a89892c174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.387733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d96b76-e7f5-41e7-93a5-30a89892c174 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.398560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1235/2476] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:16.403205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46aec-0d29-432c-a30f-ea2758ebd723 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.415356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46aec-0d29-432c-a30f-ea2758ebd723 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.422021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc05e8d-2dfd-4c2d-a01b-6c2b3455d837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.422919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc05e8d-2dfd-4c2d-a01b-6c2b3455d837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.423195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc05e8d-2dfd-4c2d-a01b-6c2b3455d837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.423510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc05e8d-2dfd-4c2d-a01b-6c2b3455d837 admin admin] PATH_INFO: `/v3/projects/d26c857c3cfd4221b87d0f9ea5816d53/users/6e14a08c1e5e400a92c4629ff1777a8e/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.439148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc05e8d-2dfd-4c2d-a01b-6c2b3455d837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d26c857c3cfd4221b87d0f9ea5816d53, user_id=6e14a08c1e5e400a92c4629ff1777a8e, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.441878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc05e8d-2dfd-4c2d-a01b-6c2b3455d837 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.456594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1242/2477] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:16 2022] PUT /identity/v3/projects/d26c857c3cfd4221b87d0f9ea5816d53/users/6e14a08c1e5e400a92c4629ff1777a8e/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:42:16.467537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec5390-f1f1-493e-9515-7eb6fa1011a2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.467839 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec5390-f1f1-493e-9515-7eb6fa1011a2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.468116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec5390-f1f1-493e-9515-7eb6fa1011a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.496021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-feec5390-f1f1-493e-9515-7eb6fa1011a2 None None] MFA Rules not processed for user `6e14a08c1e5e400a92c4629ff1777a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:16.497351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-feec5390-f1f1-493e-9515-7eb6fa1011a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:16.570369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1236/2478] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:16 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:42:16.575362 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc05e8d-2dfd-4c2d-a01b-6c2b3455d837 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.647991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:16.651882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc05e8d-2dfd-4c2d-a01b-6c2b3455d837 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.657336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16b46-b223-405f-baa0-1dae6c919987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.658018 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16b46-b223-405f-baa0-1dae6c919987 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.658662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16b46-b223-405f-baa0-1dae6c919987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.658922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e16b46-b223-405f-baa0-1dae6c919987 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.659707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e16b46-b223-405f-baa0-1dae6c919987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.661685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e16b46-b223-405f-baa0-1dae6c919987 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.691175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1243/2479] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:16 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:16.695534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec5390-f1f1-493e-9515-7eb6fa1011a2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.706814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec5390-f1f1-493e-9515-7eb6fa1011a2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.764007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1306e8b-c827-455c-9245-6425288eae45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.764594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1306e8b-c827-455c-9245-6425288eae45 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.764853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1306e8b-c827-455c-9245-6425288eae45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.765144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1306e8b-c827-455c-9245-6425288eae45 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.765826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1306e8b-c827-455c-9245-6425288eae45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.767775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1306e8b-c827-455c-9245-6425288eae45 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.787966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1237/2480] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:16 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:16.792400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16b46-b223-405f-baa0-1dae6c919987 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.802590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e16b46-b223-405f-baa0-1dae6c919987 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.807915 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210a59d-e579-4195-8769-47c14d4323cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.808598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210a59d-e579-4195-8769-47c14d4323cc admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.808874 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210a59d-e579-4195-8769-47c14d4323cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.809147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210a59d-e579-4195-8769-47c14d4323cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.809867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210a59d-e579-4195-8769-47c14d4323cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.811499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210a59d-e579-4195-8769-47c14d4323cc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.822436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1244/2481] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:16.826942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1306e8b-c827-455c-9245-6425288eae45 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.837303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1306e8b-c827-455c-9245-6425288eae45 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.842896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d48531-59b8-4827-bb90-ea77a59b3cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.843813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d48531-59b8-4827-bb90-ea77a59b3cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.844090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d48531-59b8-4827-bb90-ea77a59b3cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.844367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d48531-59b8-4827-bb90-ea77a59b3cc5 admin admin] PATH_INFO: `/v3/projects/f2ec61047d20416e9b49301e745d1743/users/9875ac9a02a64d518c70beb02ac036f3/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.860944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d48531-59b8-4827-bb90-ea77a59b3cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ec61047d20416e9b49301e745d1743, user_id=9875ac9a02a64d518c70beb02ac036f3, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.863676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d48531-59b8-4827-bb90-ea77a59b3cc5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.880357 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1238/2482] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:16 2022] PUT /identity/v3/projects/f2ec61047d20416e9b49301e745d1743/users/9875ac9a02a64d518c70beb02ac036f3/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:16.884582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210a59d-e579-4195-8769-47c14d4323cc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.894796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210a59d-e579-4195-8769-47c14d4323cc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.955271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb8ede-ed2e-4824-9f02-f4f0cf4658c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:16.956283 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8ede-ed2e-4824-9f02-f4f0cf4658c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:16.956674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8ede-ed2e-4824-9f02-f4f0cf4658c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:16.957084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb8ede-ed2e-4824-9f02-f4f0cf4658c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:16.958206 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb8ede-ed2e-4824-9f02-f4f0cf4658c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:16.960694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb8ede-ed2e-4824-9f02-f4f0cf4658c5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:16.972613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1245/2483] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:16.977015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d48531-59b8-4827-bb90-ea77a59b3cc5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:16.982804 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:16.991107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d48531-59b8-4827-bb90-ea77a59b3cc5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:16.998837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decd3a6-ad78-4c2b-aee0-245f898bca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:17.000010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decd3a6-ad78-4c2b-aee0-245f898bca85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:17.000291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decd3a6-ad78-4c2b-aee0-245f898bca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:17.000755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decd3a6-ad78-4c2b-aee0-245f898bca85 admin admin] PATH_INFO: `/v3/projects/f2ec61047d20416e9b49301e745d1743/users/9875ac9a02a64d518c70beb02ac036f3/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:17.003260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8decd3a6-ad78-4c2b-aee0-245f898bca85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ec61047d20416e9b49301e745d1743, user_id=9875ac9a02a64d518c70beb02ac036f3, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:17.006142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8decd3a6-ad78-4c2b-aee0-245f898bca85 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:17.017528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1239/2484] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:16 2022] PUT /identity/v3/projects/f2ec61047d20416e9b49301e745d1743/users/9875ac9a02a64d518c70beb02ac036f3/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:17.029494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19daa9d-06af-4de6-a9f5-51bd5dedaf85 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:17.029784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19daa9d-06af-4de6-a9f5-51bd5dedaf85 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:17.030085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19daa9d-06af-4de6-a9f5-51bd5dedaf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:17.059595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f19daa9d-06af-4de6-a9f5-51bd5dedaf85 None None] MFA Rules not processed for user `9875ac9a02a64d518c70beb02ac036f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:17.060806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f19daa9d-06af-4de6-a9f5-51bd5dedaf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:17.140662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1246/2485] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:17 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:17.152048 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decd3a6-ad78-4c2b-aee0-245f898bca85 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:17.153828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8decd3a6-ad78-4c2b-aee0-245f898bca85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:17.211958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decd3a6-ad78-4c2b-aee0-245f898bca85 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:17.216382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef31ac-f721-4c8f-9623-0d301426e8a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:17.217045 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef31ac-f721-4c8f-9623-0d301426e8a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:17.217353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef31ac-f721-4c8f-9623-0d301426e8a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:17.217647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef31ac-f721-4c8f-9623-0d301426e8a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:17.231709 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef31ac-f721-4c8f-9623-0d301426e8a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:17.233162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef31ac-f721-4c8f-9623-0d301426e8a9 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:17.242986 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1240/2486] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:17 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:18.671878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19daa9d-06af-4de6-a9f5-51bd5dedaf85 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:18.724039 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:18.736290 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19daa9d-06af-4de6-a9f5-51bd5dedaf85 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:18.741470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:18.742366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:18.742646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:18.742917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] PATH_INFO: `/v3/users/6e14a08c1e5e400a92c4629ff1777a8e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:18.744137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e14a08c1e5e400a92c4629ff1777a8e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:18.746178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:18.795661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e14a08c1e5e400a92c4629ff1777a8e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:18.796339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e14a08c1e5e400a92c4629ff1777a8e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:18.806160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e14a08c1e5e400a92c4629ff1777a8e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:18.825649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e14a08c1e5e400a92c4629ff1777a8e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:18.826410 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e14a08c1e5e400a92c4629ff1777a8e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:18.838831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1247/2487] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:18 2022] DELETE /identity/v3/users/6e14a08c1e5e400a92c4629ff1777a8e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:18.843798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2a469-0ac3-4a02-a57e-5b434ad4ec88 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:18.844286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2a469-0ac3-4a02-a57e-5b434ad4ec88 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:18.844396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2a469-0ac3-4a02-a57e-5b434ad4ec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:18.863057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-deb2a469-0ac3-4a02-a57e-5b434ad4ec88 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:18.864330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-deb2a469-0ac3-4a02-a57e-5b434ad4ec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:18.924365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1241/2488] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:42:18.934778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:18.937084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:19.007190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f776-692a-4a6e-ac0d-83df5ae29072 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:19.011179 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e761b-5bc6-4f00-9eb0-eb4c56b079b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:19.011908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e761b-5bc6-4f00-9eb0-eb4c56b079b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:19.012184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e761b-5bc6-4f00-9eb0-eb4c56b079b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:19.012452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e761b-5bc6-4f00-9eb0-eb4c56b079b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:19.026593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e761b-5bc6-4f00-9eb0-eb4c56b079b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:19.028734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e761b-5bc6-4f00-9eb0-eb4c56b079b4 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:19.038676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1248/2489] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:19.328303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2a469-0ac3-4a02-a57e-5b434ad4ec88 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:19.329788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-deb2a469-0ac3-4a02-a57e-5b434ad4ec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:19.343588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2a469-0ac3-4a02-a57e-5b434ad4ec88 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:19.349018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:19.349938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:19.350105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:19.350389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] PATH_INFO: `/v3/projects/d26c857c3cfd4221b87d0f9ea5816d53` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:19.352266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d26c857c3cfd4221b87d0f9ea5816d53)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:19.354094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:19.402942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd26c857c3cfd4221b87d0f9ea5816d53', 'request_id': 'req-eda2743c-de20-4635-874f-6ca3d4dcf33c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.411522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd26c857c3cfd4221b87d0f9ea5816d53', 'request_id': 'req-eda2743c-de20-4635-874f-6ca3d4dcf33c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.418384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd26c857c3cfd4221b87d0f9ea5816d53', 'request_id': 'req-eda2743c-de20-4635-874f-6ca3d4dcf33c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.428849 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] The token cache is being invalidate because project d26c857c3cfd4221b87d0f9ea5816d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:19.429140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.429784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.431370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1242/2490] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:19 2022] DELETE /identity/v3/projects/d26c857c3cfd4221b87d0f9ea5816d53 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:19.436242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e761b-5bc6-4f00-9eb0-eb4c56b079b4 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:19.451613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e761b-5bc6-4f00-9eb0-eb4c56b079b4 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:19.518181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:19.519071 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:19.519451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:19.519806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] PATH_INFO: `/v3/users/9875ac9a02a64d518c70beb02ac036f3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:19.521199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9875ac9a02a64d518c70beb02ac036f3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:19.523393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:19.540496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:19.580163 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9875ac9a02a64d518c70beb02ac036f3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.580848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9875ac9a02a64d518c70beb02ac036f3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.589728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9875ac9a02a64d518c70beb02ac036f3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.602147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9875ac9a02a64d518c70beb02ac036f3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.602839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db2c1ade-856f-4538-9ac8-f73dc15654e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9875ac9a02a64d518c70beb02ac036f3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:19.615489 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1249/2491] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:19 2022] DELETE /identity/v3/users/9875ac9a02a64d518c70beb02ac036f3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:19.901016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:19.902608 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:19.923317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2743c-de20-4635-874f-6ca3d4dcf33c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:19.937661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:19.986346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:19.987230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:19.987544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:19.987821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] PATH_INFO: `/v3/projects/f2ec61047d20416e9b49301e745d1743` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:19.989127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2ec61047d20416e9b49301e745d1743)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:19.991202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.049449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2ec61047d20416e9b49301e745d1743', 'request_id': 'req-79537f25-7946-4932-b42f-1b8dc5de84df'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:20.060212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2ec61047d20416e9b49301e745d1743', 'request_id': 'req-79537f25-7946-4932-b42f-1b8dc5de84df'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:20.066593 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2ec61047d20416e9b49301e745d1743', 'request_id': 'req-79537f25-7946-4932-b42f-1b8dc5de84df'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:20.077367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] The token cache is being invalidate because project f2ec61047d20416e9b49301e745d1743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:20.077729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:20.078380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-79537f25-7946-4932-b42f-1b8dc5de84df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:20.079985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1243/2492] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:19 2022] DELETE /identity/v3/projects/f2ec61047d20416e9b49301e745d1743 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:42:20.085103 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac96192-bd84-441e-bd1c-a3a71eed494f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.085574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac96192-bd84-441e-bd1c-a3a71eed494f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.085679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac96192-bd84-441e-bd1c-a3a71eed494f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.105862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5ac96192-bd84-441e-bd1c-a3a71eed494f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:20.107070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5ac96192-bd84-441e-bd1c-a3a71eed494f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:20.170085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1250/2493] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:20.177922 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac8446-7fdd-4e50-9ba5-cd3c329b6d99 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.178217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac8446-7fdd-4e50-9ba5-cd3c329b6d99 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.178499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac8446-7fdd-4e50-9ba5-cd3c329b6d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.197067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-8bac8446-7fdd-4e50-9ba5-cd3c329b6d99 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:20.198270 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8bac8446-7fdd-4e50-9ba5-cd3c329b6d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:20.206871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1244/2494] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:20.211567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac96192-bd84-441e-bd1c-a3a71eed494f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.226738 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac96192-bd84-441e-bd1c-a3a71eed494f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.233651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ce403-fa93-42e2-abaa-772eeaa3e6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.234371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ce403-fa93-42e2-abaa-772eeaa3e6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.234597 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ce403-fa93-42e2-abaa-772eeaa3e6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.234850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ce403-fa93-42e2-abaa-772eeaa3e6c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.235615 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ce403-fa93-42e2-abaa-772eeaa3e6c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:42:20.235919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ce403-fa93-42e2-abaa-772eeaa3e6c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.237589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ce403-fa93-42e2-abaa-772eeaa3e6c2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.250211 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1251/2495] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:42:20.255200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac8446-7fdd-4e50-9ba5-cd3c329b6d99 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.265892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac8446-7fdd-4e50-9ba5-cd3c329b6d99 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.270635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeb8b8-9a13-4248-b6c8-ac578d4dd7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.271347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeb8b8-9a13-4248-b6c8-ac578d4dd7cc admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.271636 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeb8b8-9a13-4248-b6c8-ac578d4dd7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.271911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeb8b8-9a13-4248-b6c8-ac578d4dd7cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.272619 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbeb8b8-9a13-4248-b6c8-ac578d4dd7cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.274545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbeb8b8-9a13-4248-b6c8-ac578d4dd7cc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.303717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1245/2496] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:20 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:42:20.308130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ce403-fa93-42e2-abaa-772eeaa3e6c2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.320517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ce403-fa93-42e2-abaa-772eeaa3e6c2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.380150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b4f3d-3cab-4106-a57b-f53b706f3872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.380825 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b4f3d-3cab-4106-a57b-f53b706f3872 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.381109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b4f3d-3cab-4106-a57b-f53b706f3872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.381381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b4f3d-3cab-4106-a57b-f53b706f3872 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.382080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b4f3d-3cab-4106-a57b-f53b706f3872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.384075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b4f3d-3cab-4106-a57b-f53b706f3872 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.410692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1252/2497] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:20 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:20.412019 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeb8b8-9a13-4248-b6c8-ac578d4dd7cc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.423031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeb8b8-9a13-4248-b6c8-ac578d4dd7cc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.427923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc40bc9-408c-473e-90a0-74a34da18827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.428585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc40bc9-408c-473e-90a0-74a34da18827 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.428867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc40bc9-408c-473e-90a0-74a34da18827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.429149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc40bc9-408c-473e-90a0-74a34da18827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.429874 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc40bc9-408c-473e-90a0-74a34da18827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.431530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc40bc9-408c-473e-90a0-74a34da18827 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.442324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1246/2498] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:20.446326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b4f3d-3cab-4106-a57b-f53b706f3872 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.456614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b4f3d-3cab-4106-a57b-f53b706f3872 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.461653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa49403-8d87-4900-92c5-89bd899217f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.462315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa49403-8d87-4900-92c5-89bd899217f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.462599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa49403-8d87-4900-92c5-89bd899217f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.462876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa49403-8d87-4900-92c5-89bd899217f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.463640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa49403-8d87-4900-92c5-89bd899217f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.465332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa49403-8d87-4900-92c5-89bd899217f8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.470469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:20.477398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1253/2499] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:20.481319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc40bc9-408c-473e-90a0-74a34da18827 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.491702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc40bc9-408c-473e-90a0-74a34da18827 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.496716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9003cc2-e958-4789-9821-cbb8bdbac3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.497746 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9003cc2-e958-4789-9821-cbb8bdbac3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.498057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9003cc2-e958-4789-9821-cbb8bdbac3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.498294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9003cc2-e958-4789-9821-cbb8bdbac3a6 admin admin] PATH_INFO: `/v3/projects/4c0d5513dd404eb88e170d6b304cddd0/users/baa722bf062746ddac1a0c6cbbbf4790/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.514716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9003cc2-e958-4789-9821-cbb8bdbac3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c0d5513dd404eb88e170d6b304cddd0, user_id=baa722bf062746ddac1a0c6cbbbf4790, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.517446 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9003cc2-e958-4789-9821-cbb8bdbac3a6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.532767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1247/2500] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:20 2022] PUT /identity/v3/projects/4c0d5513dd404eb88e170d6b304cddd0/users/baa722bf062746ddac1a0c6cbbbf4790/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:42:20.545331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a000f-7be1-42b6-9077-e358355add7a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.545610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a000f-7be1-42b6-9077-e358355add7a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.545891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a000f-7be1-42b6-9077-e358355add7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.572195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-749a000f-7be1-42b6-9077-e358355add7a None None] MFA Rules not processed for user `baa722bf062746ddac1a0c6cbbbf4790`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:20.573419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-749a000f-7be1-42b6-9077-e358355add7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:20.644231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1254/2501] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:20.649273 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9003cc2-e958-4789-9821-cbb8bdbac3a6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.700103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:20.725856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9003cc2-e958-4789-9821-cbb8bdbac3a6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.733592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07d466-84e1-4788-8dc4-7b954a170fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.734262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07d466-84e1-4788-8dc4-7b954a170fce admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.734533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07d466-84e1-4788-8dc4-7b954a170fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.734801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07d466-84e1-4788-8dc4-7b954a170fce admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.735528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07d466-84e1-4788-8dc4-7b954a170fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.737447 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07d466-84e1-4788-8dc4-7b954a170fce admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.766801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1248/2502] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:20 2022] POST /identity/v3/projects => generated 369 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:42:20.771574 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a000f-7be1-42b6-9077-e358355add7a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.788891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a000f-7be1-42b6-9077-e358355add7a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.849844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6513e-29de-4d90-9a21-1b94bc9becfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.850515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6513e-29de-4d90-9a21-1b94bc9becfc admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.850797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6513e-29de-4d90-9a21-1b94bc9becfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.851074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6513e-29de-4d90-9a21-1b94bc9becfc admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.851799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6513e-29de-4d90-9a21-1b94bc9becfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.853734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6513e-29de-4d90-9a21-1b94bc9becfc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.875863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1255/2503] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:20 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:20.879799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07d466-84e1-4788-8dc4-7b954a170fce admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.890639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07d466-84e1-4788-8dc4-7b954a170fce admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.896205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbb0f1-a043-444b-8b29-af0cf270258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.896860 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb0f1-a043-444b-8b29-af0cf270258c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.897141 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb0f1-a043-444b-8b29-af0cf270258c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.897433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb0f1-a043-444b-8b29-af0cf270258c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.898155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbb0f1-a043-444b-8b29-af0cf270258c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.899841 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbb0f1-a043-444b-8b29-af0cf270258c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.910877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1249/2504] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:20.914989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6513e-29de-4d90-9a21-1b94bc9becfc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.927239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6513e-29de-4d90-9a21-1b94bc9becfc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:20.932794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c6a53-4e8f-48d2-ac33-39bbd5ab5714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:20.933689 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c6a53-4e8f-48d2-ac33-39bbd5ab5714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:20.933970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c6a53-4e8f-48d2-ac33-39bbd5ab5714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:20.934243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c6a53-4e8f-48d2-ac33-39bbd5ab5714 admin admin] PATH_INFO: `/v3/projects/915e84baf8184311854afabcb5ef49bb/users/2045abfe20854e3baad52f768359cd6c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:20.950944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c6a53-4e8f-48d2-ac33-39bbd5ab5714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=915e84baf8184311854afabcb5ef49bb, user_id=2045abfe20854e3baad52f768359cd6c, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:20.953672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c6a53-4e8f-48d2-ac33-39bbd5ab5714 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:20.969628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1256/2505] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:20 2022] PUT /identity/v3/projects/915e84baf8184311854afabcb5ef49bb/users/2045abfe20854e3baad52f768359cd6c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:20.973856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbb0f1-a043-444b-8b29-af0cf270258c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:20.984276 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbb0f1-a043-444b-8b29-af0cf270258c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:21.049571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c20b06-fefb-4b3b-a411-2a0a07964997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:21.050154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c20b06-fefb-4b3b-a411-2a0a07964997 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:21.050435 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c20b06-fefb-4b3b-a411-2a0a07964997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:21.050710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c20b06-fefb-4b3b-a411-2a0a07964997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:21.051464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c20b06-fefb-4b3b-a411-2a0a07964997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:21.053117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c20b06-fefb-4b3b-a411-2a0a07964997 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:21.063417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1250/2506] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:42:21.067769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c6a53-4e8f-48d2-ac33-39bbd5ab5714 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:21.078470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c6a53-4e8f-48d2-ac33-39bbd5ab5714 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:21.083250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d212052-5c04-4871-b9f1-7a02e316d57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:21.084171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d212052-5c04-4871-b9f1-7a02e316d57f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:21.084425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d212052-5c04-4871-b9f1-7a02e316d57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:21.084747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d212052-5c04-4871-b9f1-7a02e316d57f admin admin] PATH_INFO: `/v3/projects/915e84baf8184311854afabcb5ef49bb/users/2045abfe20854e3baad52f768359cd6c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:21.086698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d212052-5c04-4871-b9f1-7a02e316d57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=915e84baf8184311854afabcb5ef49bb, user_id=2045abfe20854e3baad52f768359cd6c, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:21.089370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d212052-5c04-4871-b9f1-7a02e316d57f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:21.104303 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1257/2507] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:21 2022] PUT /identity/v3/projects/915e84baf8184311854afabcb5ef49bb/users/2045abfe20854e3baad52f768359cd6c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:21.116853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43331f1a-3e6e-4957-98f1-12c8b293050c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:21.117321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43331f1a-3e6e-4957-98f1-12c8b293050c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:21.117637 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43331f1a-3e6e-4957-98f1-12c8b293050c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:21.152462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-43331f1a-3e6e-4957-98f1-12c8b293050c None None] MFA Rules not processed for user `2045abfe20854e3baad52f768359cd6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:21.154076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-43331f1a-3e6e-4957-98f1-12c8b293050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:21.228168 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1251/2508] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:21 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:21.239913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d212052-5c04-4871-b9f1-7a02e316d57f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:21.241742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8d212052-5c04-4871-b9f1-7a02e316d57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:21.287171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:21.302005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d212052-5c04-4871-b9f1-7a02e316d57f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:21.306446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f94bc8-b579-4d88-9d21-bbabc2bad043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:21.307138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f94bc8-b579-4d88-9d21-bbabc2bad043 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:21.307457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f94bc8-b579-4d88-9d21-bbabc2bad043 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:21.307735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f94bc8-b579-4d88-9d21-bbabc2bad043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:21.319097 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f94bc8-b579-4d88-9d21-bbabc2bad043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:21.320616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f94bc8-b579-4d88-9d21-bbabc2bad043 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:21.331355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1258/2509] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:21 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:21.377962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43331f1a-3e6e-4957-98f1-12c8b293050c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:21.379706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-43331f1a-3e6e-4957-98f1-12c8b293050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:21.393178 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43331f1a-3e6e-4957-98f1-12c8b293050c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:21.397327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2520297-c61c-4495-9795-f49abea47ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:21.397991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2520297-c61c-4495-9795-f49abea47ced service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:21.398277 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2520297-c61c-4495-9795-f49abea47ced service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:21.398558 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2520297-c61c-4495-9795-f49abea47ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:21.434310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:21.458566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2520297-c61c-4495-9795-f49abea47ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:21.460058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2520297-c61c-4495-9795-f49abea47ced service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:21.468932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1252/2510] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:24.848339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f94bc8-b579-4d88-9d21-bbabc2bad043 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:24.914217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f94bc8-b579-4d88-9d21-bbabc2bad043 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:24.919850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:24.920715 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:24.920993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:24.921272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] PATH_INFO: `/v3/users/baa722bf062746ddac1a0c6cbbbf4790` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:24.922496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa722bf062746ddac1a0c6cbbbf4790)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:24.924711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:24.977535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa722bf062746ddac1a0c6cbbbf4790'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:24.978208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa722bf062746ddac1a0c6cbbbf4790'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:24.987639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa722bf062746ddac1a0c6cbbbf4790'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.003501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa722bf062746ddac1a0c6cbbbf4790'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.004226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa722bf062746ddac1a0c6cbbbf4790'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.016145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1259/2511] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:24 2022] DELETE /identity/v3/users/baa722bf062746ddac1a0c6cbbbf4790 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:42:25.021004 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0096-6183-4d37-b6df-a5bcb92b3c2b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:25.021483 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0096-6183-4d37-b6df-a5bcb92b3c2b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:25.021588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0096-6183-4d37-b6df-a5bcb92b3c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:25.040812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eb8d0096-6183-4d37-b6df-a5bcb92b3c2b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:25.043021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eb8d0096-6183-4d37-b6df-a5bcb92b3c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:25.115548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1253/2512] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:25.126288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:25.127826 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:25.189082 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88283a6-e98d-44a9-9808-b949e7999c38 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:25.193413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4328251-fd33-4f78-9a5e-6495d996cd90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:25.194063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4328251-fd33-4f78-9a5e-6495d996cd90 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:25.194344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4328251-fd33-4f78-9a5e-6495d996cd90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:25.194616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4328251-fd33-4f78-9a5e-6495d996cd90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:25.205871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4328251-fd33-4f78-9a5e-6495d996cd90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:25.207669 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4328251-fd33-4f78-9a5e-6495d996cd90 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:25.217841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1260/2513] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:25.504901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d0096-6183-4d37-b6df-a5bcb92b3c2b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:25.506426 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eb8d0096-6183-4d37-b6df-a5bcb92b3c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:25.518934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d0096-6183-4d37-b6df-a5bcb92b3c2b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:25.524781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:25.525634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:25.525910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:25.526214 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] PATH_INFO: `/v3/projects/4c0d5513dd404eb88e170d6b304cddd0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:25.527545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c0d5513dd404eb88e170d6b304cddd0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:25.529486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:25.577957 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c0d5513dd404eb88e170d6b304cddd0', 'request_id': 'req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.587052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c0d5513dd404eb88e170d6b304cddd0', 'request_id': 'req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.593779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c0d5513dd404eb88e170d6b304cddd0', 'request_id': 'req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.605581 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] The token cache is being invalidate because project 4c0d5513dd404eb88e170d6b304cddd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:25.605845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.606517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.608373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1254/2514] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:25 2022] DELETE /identity/v3/projects/4c0d5513dd404eb88e170d6b304cddd0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:42:25.612409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4328251-fd33-4f78-9a5e-6495d996cd90 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:25.622706 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4328251-fd33-4f78-9a5e-6495d996cd90 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:25.626654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:25.685038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:25.685918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:25.686197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:25.686471 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] PATH_INFO: `/v3/users/2045abfe20854e3baad52f768359cd6c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:25.687695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2045abfe20854e3baad52f768359cd6c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:25.689625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:25.734496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2045abfe20854e3baad52f768359cd6c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.735158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2045abfe20854e3baad52f768359cd6c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.743900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2045abfe20854e3baad52f768359cd6c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.755925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2045abfe20854e3baad52f768359cd6c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.756547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4a06d3cf-b70c-4917-9fbf-8e4b66cfb733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2045abfe20854e3baad52f768359cd6c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:25.768457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1261/2515] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:25 2022] DELETE /identity/v3/users/2045abfe20854e3baad52f768359cd6c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:26.077775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:26.079325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:26.092518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0a1fe-b8db-4ade-ba77-2f9ca2990b5b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:26.153523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:26.154412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.154675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.154942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] PATH_INFO: `/v3/projects/915e84baf8184311854afabcb5ef49bb` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.156184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=915e84baf8184311854afabcb5ef49bb)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:26.158113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:26.159611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:26.209351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '915e84baf8184311854afabcb5ef49bb', 'request_id': 'req-a3b33524-a865-42a6-96db-b626c1dddf11'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:26.219128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '915e84baf8184311854afabcb5ef49bb', 'request_id': 'req-a3b33524-a865-42a6-96db-b626c1dddf11'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:26.226083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '915e84baf8184311854afabcb5ef49bb', 'request_id': 'req-a3b33524-a865-42a6-96db-b626c1dddf11'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:26.237278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] The token cache is being invalidate because project 915e84baf8184311854afabcb5ef49bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:26.237565 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:26.238238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-a3b33524-a865-42a6-96db-b626c1dddf11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:26.240064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1255/2516] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:26 2022] DELETE /identity/v3/projects/915e84baf8184311854afabcb5ef49bb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:26.245171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f930819-e120-475d-b450-a9e5c860c2fc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.245666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f930819-e120-475d-b450-a9e5c860c2fc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.245772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f930819-e120-475d-b450-a9e5c860c2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.264373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8f930819-e120-475d-b450-a9e5c860c2fc None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:26.265649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8f930819-e120-475d-b450-a9e5c860c2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:26.332031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1262/2517] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:26.340508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955b8a3-9d60-454b-a917-7ae6dcd0a355 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.340793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955b8a3-9d60-454b-a917-7ae6dcd0a355 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.341064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955b8a3-9d60-454b-a917-7ae6dcd0a355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.361323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a955b8a3-9d60-454b-a917-7ae6dcd0a355 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:26.362534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a955b8a3-9d60-454b-a917-7ae6dcd0a355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:26.372904 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1256/2518] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:26.377405 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f930819-e120-475d-b450-a9e5c860c2fc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:26.388131 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f930819-e120-475d-b450-a9e5c860c2fc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:26.393735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281141e3-07e7-4e83-8a45-de251243a559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:26.394422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281141e3-07e7-4e83-8a45-de251243a559 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.394720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281141e3-07e7-4e83-8a45-de251243a559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.395001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281141e3-07e7-4e83-8a45-de251243a559 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.395779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281141e3-07e7-4e83-8a45-de251243a559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:42:26.396095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281141e3-07e7-4e83-8a45-de251243a559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:26.397712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281141e3-07e7-4e83-8a45-de251243a559 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:26.410056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1263/2519] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:42:26.416592 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955b8a3-9d60-454b-a917-7ae6dcd0a355 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:26.429107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955b8a3-9d60-454b-a917-7ae6dcd0a355 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:26.436640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89d517-c369-4c6b-8ca3-daf061d4381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:26.437605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89d517-c369-4c6b-8ca3-daf061d4381d admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.437991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89d517-c369-4c6b-8ca3-daf061d4381d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.438306 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89d517-c369-4c6b-8ca3-daf061d4381d admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.439363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89d517-c369-4c6b-8ca3-daf061d4381d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:26.442400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89d517-c369-4c6b-8ca3-daf061d4381d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:26.475585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1257/2520] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:26 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:26.479871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281141e3-07e7-4e83-8a45-de251243a559 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:26.490042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281141e3-07e7-4e83-8a45-de251243a559 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:26.498959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:26.548796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca9516-6f12-4281-ab55-6d0a59deace5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:26.549407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca9516-6f12-4281-ab55-6d0a59deace5 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.549667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca9516-6f12-4281-ab55-6d0a59deace5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.549924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca9516-6f12-4281-ab55-6d0a59deace5 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.550617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca9516-6f12-4281-ab55-6d0a59deace5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:26.552667 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca9516-6f12-4281-ab55-6d0a59deace5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:26.580014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1264/2521] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:26 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:26.584882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89d517-c369-4c6b-8ca3-daf061d4381d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:26.596132 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89d517-c369-4c6b-8ca3-daf061d4381d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:26.601943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16534439-b470-45e7-aa8d-1d89654dc2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:26.602606 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16534439-b470-45e7-aa8d-1d89654dc2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.602890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16534439-b470-45e7-aa8d-1d89654dc2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.603168 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16534439-b470-45e7-aa8d-1d89654dc2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.603931 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16534439-b470-45e7-aa8d-1d89654dc2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:26.605560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16534439-b470-45e7-aa8d-1d89654dc2d9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:26.618066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1258/2522] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:26.622806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca9516-6f12-4281-ab55-6d0a59deace5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:26.638677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca9516-6f12-4281-ab55-6d0a59deace5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:26.646803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab8bea-7b19-4500-be56-d17f4c1d5f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:26.647504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab8bea-7b19-4500-be56-d17f4c1d5f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.647789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab8bea-7b19-4500-be56-d17f4c1d5f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.648058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab8bea-7b19-4500-be56-d17f4c1d5f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.648789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab8bea-7b19-4500-be56-d17f4c1d5f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:26.650409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab8bea-7b19-4500-be56-d17f4c1d5f66 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:26.661470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1265/2523] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:26.666837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16534439-b470-45e7-aa8d-1d89654dc2d9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:26.680479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16534439-b470-45e7-aa8d-1d89654dc2d9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:26.685449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b1b9d-4561-40e9-b461-e3d3024a62b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:26.686324 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b1b9d-4561-40e9-b461-e3d3024a62b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.686605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b1b9d-4561-40e9-b461-e3d3024a62b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.686877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b1b9d-4561-40e9-b461-e3d3024a62b1 admin admin] PATH_INFO: `/v3/projects/d55dc583780047919dbce09f8bd32dc3/users/27ea5fefbdf8439f9fc426acb301a494/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.701334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b1b9d-4561-40e9-b461-e3d3024a62b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55dc583780047919dbce09f8bd32dc3, user_id=27ea5fefbdf8439f9fc426acb301a494, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:26.704055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b1b9d-4561-40e9-b461-e3d3024a62b1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:26.720371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1259/2524] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:26 2022] PUT /identity/v3/projects/d55dc583780047919dbce09f8bd32dc3/users/27ea5fefbdf8439f9fc426acb301a494/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:26.731988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2029f0-f822-4399-9cce-68f2a8ff84ca None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.732309 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2029f0-f822-4399-9cce-68f2a8ff84ca None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.732599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2029f0-f822-4399-9cce-68f2a8ff84ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.762341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2e2029f0-f822-4399-9cce-68f2a8ff84ca None None] MFA Rules not processed for user `27ea5fefbdf8439f9fc426acb301a494`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:26.763698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2e2029f0-f822-4399-9cce-68f2a8ff84ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:26.830877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1266/2525] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:26.835731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b1b9d-4561-40e9-b461-e3d3024a62b1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:26.897231 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b1b9d-4561-40e9-b461-e3d3024a62b1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:26.902657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d2223-d839-4409-8f6a-8835b9e7a23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:26.903327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d2223-d839-4409-8f6a-8835b9e7a23c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:26.903609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d2223-d839-4409-8f6a-8835b9e7a23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:26.903882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d2223-d839-4409-8f6a-8835b9e7a23c admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:26.904561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d2223-d839-4409-8f6a-8835b9e7a23c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:26.906467 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d2223-d839-4409-8f6a-8835b9e7a23c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:26.934576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1260/2526] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:26 2022] POST /identity/v3/projects => generated 401 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:26.938774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2029f0-f822-4399-9cce-68f2a8ff84ca None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:26.950272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2029f0-f822-4399-9cce-68f2a8ff84ca None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:27.006925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd613c7-ac5b-46c6-9cbd-d517d57dc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:27.007649 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd613c7-ac5b-46c6-9cbd-d517d57dc869 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:27.007931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd613c7-ac5b-46c6-9cbd-d517d57dc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:27.008207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd613c7-ac5b-46c6-9cbd-d517d57dc869 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:27.008897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd613c7-ac5b-46c6-9cbd-d517d57dc869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:27.010843 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd613c7-ac5b-46c6-9cbd-d517d57dc869 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:27.033632 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1267/2527] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:26 2022] POST /identity/v3/users => generated 307 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:27.037147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d2223-d839-4409-8f6a-8835b9e7a23c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:27.048581 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d2223-d839-4409-8f6a-8835b9e7a23c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:27.053409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfe215-b336-4746-8492-bd340bba4647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:27.054073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfe215-b336-4746-8492-bd340bba4647 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:27.054354 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfe215-b336-4746-8492-bd340bba4647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:27.054629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfe215-b336-4746-8492-bd340bba4647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:27.055365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfe215-b336-4746-8492-bd340bba4647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:27.057011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfe215-b336-4746-8492-bd340bba4647 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:27.062582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:27.068570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1261/2528] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:27.073108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd613c7-ac5b-46c6-9cbd-d517d57dc869 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:27.083858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd613c7-ac5b-46c6-9cbd-d517d57dc869 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:27.088586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ddac5a-187e-4f6d-be61-d14702779066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:27.089485 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddac5a-187e-4f6d-be61-d14702779066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:27.089849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddac5a-187e-4f6d-be61-d14702779066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:27.090137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ddac5a-187e-4f6d-be61-d14702779066 admin admin] PATH_INFO: `/v3/projects/8dbd68844ea2413e8bffa84e52252601/users/5e079d60766340899f4b24a9bc3cab39/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:27.100583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:27.108057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ddac5a-187e-4f6d-be61-d14702779066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbd68844ea2413e8bffa84e52252601, user_id=5e079d60766340899f4b24a9bc3cab39, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:27.110924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ddac5a-187e-4f6d-be61-d14702779066 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:27.126280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1268/2529] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:27 2022] PUT /identity/v3/projects/8dbd68844ea2413e8bffa84e52252601/users/5e079d60766340899f4b24a9bc3cab39/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:27.131570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfe215-b336-4746-8492-bd340bba4647 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:27.142436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfe215-b336-4746-8492-bd340bba4647 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:27.203782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e104e-504f-4e21-b27a-be99e43d2b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:27.204481 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e104e-504f-4e21-b27a-be99e43d2b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:27.204770 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e104e-504f-4e21-b27a-be99e43d2b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:27.205088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e104e-504f-4e21-b27a-be99e43d2b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:27.205813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e104e-504f-4e21-b27a-be99e43d2b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:27.207475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e104e-504f-4e21-b27a-be99e43d2b66 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:27.217377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1262/2530] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:27.223412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ddac5a-187e-4f6d-be61-d14702779066 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:27.234614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ddac5a-187e-4f6d-be61-d14702779066 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:27.240236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b672f6-0772-4739-878e-306e81b06b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:27.241121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b672f6-0772-4739-878e-306e81b06b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:27.241393 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b672f6-0772-4739-878e-306e81b06b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:27.241676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b672f6-0772-4739-878e-306e81b06b66 admin admin] PATH_INFO: `/v3/projects/8dbd68844ea2413e8bffa84e52252601/users/5e079d60766340899f4b24a9bc3cab39/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:27.243809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b672f6-0772-4739-878e-306e81b06b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbd68844ea2413e8bffa84e52252601, user_id=5e079d60766340899f4b24a9bc3cab39, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:27.246479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b672f6-0772-4739-878e-306e81b06b66 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:27.257835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1269/2531] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:27 2022] PUT /identity/v3/projects/8dbd68844ea2413e8bffa84e52252601/users/5e079d60766340899f4b24a9bc3cab39/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:27.268529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce06fbd-2117-46c1-8f6d-979b39368269 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:27.268822 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce06fbd-2117-46c1-8f6d-979b39368269 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:27.269100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce06fbd-2117-46c1-8f6d-979b39368269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:27.300349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-dce06fbd-2117-46c1-8f6d-979b39368269 None None] MFA Rules not processed for user `5e079d60766340899f4b24a9bc3cab39`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:27.301872 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-dce06fbd-2117-46c1-8f6d-979b39368269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:27.370917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1263/2532] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:27 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:27.384826 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b672f6-0772-4739-878e-306e81b06b66 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:27.396239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d1b672f6-0772-4739-878e-306e81b06b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:27.470007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b672f6-0772-4739-878e-306e81b06b66 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:27.474987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da078995-adfc-4e12-a6d8-9ed97a4c6707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:27.475867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da078995-adfc-4e12-a6d8-9ed97a4c6707 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:27.476183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da078995-adfc-4e12-a6d8-9ed97a4c6707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:27.476584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da078995-adfc-4e12-a6d8-9ed97a4c6707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:27.486252 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da078995-adfc-4e12-a6d8-9ed97a4c6707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:27.487749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da078995-adfc-4e12-a6d8-9ed97a4c6707 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:27.498168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1270/2533] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:27 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:27.574511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce06fbd-2117-46c1-8f6d-979b39368269 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:27.584814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce06fbd-2117-46c1-8f6d-979b39368269 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:27.589187 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da2f82-e31a-444b-8152-87909f860e2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:27.589837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2f82-e31a-444b-8152-87909f860e2a service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:27.590121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2f82-e31a-444b-8152-87909f860e2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:27.590419 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2f82-e31a-444b-8152-87909f860e2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:27.652505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da2f82-e31a-444b-8152-87909f860e2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:27.653892 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da2f82-e31a-444b-8152-87909f860e2a service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:27.663089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1264/2534] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:31.800238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da078995-adfc-4e12-a6d8-9ed97a4c6707 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:31.866585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da078995-adfc-4e12-a6d8-9ed97a4c6707 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:31.873995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:31.875330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:31.875760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:31.876138 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] PATH_INFO: `/v3/users/27ea5fefbdf8439f9fc426acb301a494` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:31.878249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=27ea5fefbdf8439f9fc426acb301a494)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:31.880936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:31.944537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27ea5fefbdf8439f9fc426acb301a494'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:31.945487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27ea5fefbdf8439f9fc426acb301a494'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:31.956312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27ea5fefbdf8439f9fc426acb301a494'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:31.969048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27ea5fefbdf8439f9fc426acb301a494'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:31.969703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27ea5fefbdf8439f9fc426acb301a494'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:31.975129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:31.983834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1271/2535] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:31 2022] DELETE /identity/v3/users/27ea5fefbdf8439f9fc426acb301a494 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:31.989260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75705c-bbe2-4be2-9777-f5006b853a44 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:31.989749 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75705c-bbe2-4be2-9777-f5006b853a44 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:31.989866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75705c-bbe2-4be2-9777-f5006b853a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:32.008821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9a75705c-bbe2-4be2-9777-f5006b853a44 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:32.010052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9a75705c-bbe2-4be2-9777-f5006b853a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:32.024201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:32.084502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1265/2536] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:32.095198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:32.097582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:32.157763 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286d9cb-3140-42b9-b62a-8a00c09109cc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:32.161752 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8ec67-850f-47c5-8906-a98123efe0ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:32.162411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8ec67-850f-47c5-8906-a98123efe0ae service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:32.162686 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8ec67-850f-47c5-8906-a98123efe0ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:32.162956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8ec67-850f-47c5-8906-a98123efe0ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:32.174091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8ec67-850f-47c5-8906-a98123efe0ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:32.175611 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8ec67-850f-47c5-8906-a98123efe0ae service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:32.185396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1272/2537] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:32.467769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75705c-bbe2-4be2-9777-f5006b853a44 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:32.469862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9a75705c-bbe2-4be2-9777-f5006b853a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:32.484954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75705c-bbe2-4be2-9777-f5006b853a44 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:32.491153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:32.492072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:32.492386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:32.493073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] PATH_INFO: `/v3/projects/d55dc583780047919dbce09f8bd32dc3` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:32.493903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55dc583780047919dbce09f8bd32dc3)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:32.496427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:32.548550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55dc583780047919dbce09f8bd32dc3', 'request_id': 'req-f61135db-419b-40ed-a2e7-cec45e220e92'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.557560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55dc583780047919dbce09f8bd32dc3', 'request_id': 'req-f61135db-419b-40ed-a2e7-cec45e220e92'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.563944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55dc583780047919dbce09f8bd32dc3', 'request_id': 'req-f61135db-419b-40ed-a2e7-cec45e220e92'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.574401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] The token cache is being invalidate because project d55dc583780047919dbce09f8bd32dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:32.574691 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.575345 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.576914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1266/2538] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:32 2022] DELETE /identity/v3/projects/d55dc583780047919dbce09f8bd32dc3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:32.580961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8ec67-850f-47c5-8906-a98123efe0ae service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:32.593023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8ec67-850f-47c5-8906-a98123efe0ae service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:32.658718 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:32.659549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:32.659830 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:32.660101 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] PATH_INFO: `/v3/users/5e079d60766340899f4b24a9bc3cab39` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:32.661300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e079d60766340899f4b24a9bc3cab39)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:32.663234 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:32.711029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e079d60766340899f4b24a9bc3cab39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.711676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e079d60766340899f4b24a9bc3cab39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.719894 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e079d60766340899f4b24a9bc3cab39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.731497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e079d60766340899f4b24a9bc3cab39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.732127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c337307f-e446-4361-a389-edeb35f23836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e079d60766340899f4b24a9bc3cab39'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:32.744345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1273/2539] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:32 2022] DELETE /identity/v3/users/5e079d60766340899f4b24a9bc3cab39 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:33.040386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:33.042265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:33.055442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61135db-419b-40ed-a2e7-cec45e220e92 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:33.114309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:33.115184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.115499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.115773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] PATH_INFO: `/v3/projects/8dbd68844ea2413e8bffa84e52252601` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.117010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dbd68844ea2413e8bffa84e52252601)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:33.118938 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:33.165460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:33.168531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dbd68844ea2413e8bffa84e52252601', 'request_id': 'req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:33.178932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dbd68844ea2413e8bffa84e52252601', 'request_id': 'req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:33.185738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dbd68844ea2413e8bffa84e52252601', 'request_id': 'req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:33.196468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] The token cache is being invalidate because project 8dbd68844ea2413e8bffa84e52252601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:33.196776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:33.197468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-02d2162c-99f4-430e-97d9-eb2cbd8b7a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:33.199114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1267/2540] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:33 2022] DELETE /identity/v3/projects/8dbd68844ea2413e8bffa84e52252601 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:33.204247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de8058-8ca1-44ff-8f96-b2fbabb89901 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.204714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de8058-8ca1-44ff-8f96-b2fbabb89901 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.204806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de8058-8ca1-44ff-8f96-b2fbabb89901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.223166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-40de8058-8ca1-44ff-8f96-b2fbabb89901 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:33.224396 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-40de8058-8ca1-44ff-8f96-b2fbabb89901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:33.237849 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:33.288559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1274/2541] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:33.298855 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ae397-ab32-4ecb-8750-e5861b53db5b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.299165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ae397-ab32-4ecb-8750-e5861b53db5b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.299457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ae397-ab32-4ecb-8750-e5861b53db5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.320320 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-804ae397-ab32-4ecb-8750-e5861b53db5b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:33.321619 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-804ae397-ab32-4ecb-8750-e5861b53db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:33.330821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1268/2542] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:33.335196 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de8058-8ca1-44ff-8f96-b2fbabb89901 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:33.345595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de8058-8ca1-44ff-8f96-b2fbabb89901 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:33.352164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182b98f-6965-445c-a814-864958a62a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:33.352777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182b98f-6965-445c-a814-864958a62a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.353060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182b98f-6965-445c-a814-864958a62a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.353333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182b98f-6965-445c-a814-864958a62a97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.354063 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182b98f-6965-445c-a814-864958a62a97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:42:33.354366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182b98f-6965-445c-a814-864958a62a97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:33.356038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182b98f-6965-445c-a814-864958a62a97 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:33.368536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1275/2543] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:42:33.373017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ae397-ab32-4ecb-8750-e5861b53db5b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:33.382367 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ae397-ab32-4ecb-8750-e5861b53db5b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:33.387927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25de7d4-62a6-40a0-8c3d-b01f2d44e03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:33.388578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25de7d4-62a6-40a0-8c3d-b01f2d44e03c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.388882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25de7d4-62a6-40a0-8c3d-b01f2d44e03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.389152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25de7d4-62a6-40a0-8c3d-b01f2d44e03c admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.389854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25de7d4-62a6-40a0-8c3d-b01f2d44e03c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:33.391803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25de7d4-62a6-40a0-8c3d-b01f2d44e03c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:33.420916 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1269/2544] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:33 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:33.425323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182b98f-6965-445c-a814-864958a62a97 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:33.436378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182b98f-6965-445c-a814-864958a62a97 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:33.494402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405844b8-e450-4b52-b44a-9f9188d6ff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:33.495130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405844b8-e450-4b52-b44a-9f9188d6ff98 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.495450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405844b8-e450-4b52-b44a-9f9188d6ff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.495724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405844b8-e450-4b52-b44a-9f9188d6ff98 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.496417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405844b8-e450-4b52-b44a-9f9188d6ff98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:33.498355 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405844b8-e450-4b52-b44a-9f9188d6ff98 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:33.519927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1276/2545] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:33 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:33.524570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25de7d4-62a6-40a0-8c3d-b01f2d44e03c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:33.534799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25de7d4-62a6-40a0-8c3d-b01f2d44e03c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:33.539811 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7753589-ea7b-42da-8974-2f637284ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:33.540470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7753589-ea7b-42da-8974-2f637284ca40 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.540750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7753589-ea7b-42da-8974-2f637284ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.541026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7753589-ea7b-42da-8974-2f637284ca40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.542096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7753589-ea7b-42da-8974-2f637284ca40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:33.543742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7753589-ea7b-42da-8974-2f637284ca40 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:33.557886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1270/2546] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:42:33.563599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405844b8-e450-4b52-b44a-9f9188d6ff98 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:33.573966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405844b8-e450-4b52-b44a-9f9188d6ff98 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:33.578962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8ba05-4178-4e67-b8bf-b55e8a2ffc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:33.579665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8ba05-4178-4e67-b8bf-b55e8a2ffc19 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.579935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8ba05-4178-4e67-b8bf-b55e8a2ffc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.580219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8ba05-4178-4e67-b8bf-b55e8a2ffc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.580979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8ba05-4178-4e67-b8bf-b55e8a2ffc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:33.582560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8ba05-4178-4e67-b8bf-b55e8a2ffc19 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:33.592692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1277/2547] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:42:33.596982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7753589-ea7b-42da-8974-2f637284ca40 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:33.607319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7753589-ea7b-42da-8974-2f637284ca40 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:33.612135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703e454-16d6-45b4-b2c9-2941421b1a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:33.613006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703e454-16d6-45b4-b2c9-2941421b1a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.613282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703e454-16d6-45b4-b2c9-2941421b1a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.613552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703e454-16d6-45b4-b2c9-2941421b1a18 admin admin] PATH_INFO: `/v3/projects/c09f808b0aae46e1abbd8055cf207af9/users/6ef8845b70d947b4827951ec28c12655/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.628046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703e454-16d6-45b4-b2c9-2941421b1a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09f808b0aae46e1abbd8055cf207af9, user_id=6ef8845b70d947b4827951ec28c12655, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:33.630682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703e454-16d6-45b4-b2c9-2941421b1a18 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:33.646385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1271/2548] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:33 2022] PUT /identity/v3/projects/c09f808b0aae46e1abbd8055cf207af9/users/6ef8845b70d947b4827951ec28c12655/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:33.715929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510383be-e7f7-465a-a8dc-e1f81ff15742 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.716164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510383be-e7f7-465a-a8dc-e1f81ff15742 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.716382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510383be-e7f7-465a-a8dc-e1f81ff15742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.744496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-510383be-e7f7-465a-a8dc-e1f81ff15742 None None] MFA Rules not processed for user `6ef8845b70d947b4827951ec28c12655`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:33.745690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-510383be-e7f7-465a-a8dc-e1f81ff15742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:33.815760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1278/2549] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:33.827540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703e454-16d6-45b4-b2c9-2941421b1a18 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:33.829587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1703e454-16d6-45b4-b2c9-2941421b1a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:33.889836 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703e454-16d6-45b4-b2c9-2941421b1a18 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:33.894002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8ec9b-816b-4a03-9b39-ab7644d795bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:33.894736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8ec9b-816b-4a03-9b39-ab7644d795bc service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:33.895094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8ec9b-816b-4a03-9b39-ab7644d795bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:33.895472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8ec9b-816b-4a03-9b39-ab7644d795bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:33.904283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8ec9b-816b-4a03-9b39-ab7644d795bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:33.905730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8ec9b-816b-4a03-9b39-ab7644d795bc service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:33.914960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1272/2550] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:43.174147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510383be-e7f7-465a-a8dc-e1f81ff15742 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:43.175829 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-510383be-e7f7-465a-a8dc-e1f81ff15742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:43.192173 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:43.251208 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510383be-e7f7-465a-a8dc-e1f81ff15742 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:43.257059 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aaea5-8600-4ede-8ccf-c9a2b33e61b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:43.257707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aaea5-8600-4ede-8ccf-c9a2b33e61b7 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:43.257990 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aaea5-8600-4ede-8ccf-c9a2b33e61b7 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:43.258265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aaea5-8600-4ede-8ccf-c9a2b33e61b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:43.260357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5f2aaea5-8600-4ede-8ccf-c9a2b33e61b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:43.272232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2aaea5-8600-4ede-8ccf-c9a2b33e61b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:43.273828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2aaea5-8600-4ede-8ccf-c9a2b33e61b7 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:43.284589 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1279/2551] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:43.419686 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8ec9b-816b-4a03-9b39-ab7644d795bc service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:43.486577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8ec9b-816b-4a03-9b39-ab7644d795bc service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:43.491202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:43.492101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:43.492377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:43.492649 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] PATH_INFO: `/v3/users/6ef8845b70d947b4827951ec28c12655` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:43.493840 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef8845b70d947b4827951ec28c12655)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:43.495814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:43.509242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:43.552500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef8845b70d947b4827951ec28c12655'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:43.553328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef8845b70d947b4827951ec28c12655'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:43.565007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef8845b70d947b4827951ec28c12655'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:43.565710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef8845b70d947b4827951ec28c12655'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:43.577496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef8845b70d947b4827951ec28c12655'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:43.590742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1273/2552] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:43 2022] DELETE /identity/v3/users/6ef8845b70d947b4827951ec28c12655 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:43.594959 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73735a-f050-4e84-b3d6-dbb2a94b106e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:43.595249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73735a-f050-4e84-b3d6-dbb2a94b106e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:43.595551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73735a-f050-4e84-b3d6-dbb2a94b106e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:43.614406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ea73735a-f050-4e84-b3d6-dbb2a94b106e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:43.615641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ea73735a-f050-4e84-b3d6-dbb2a94b106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:43.681337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1280/2553] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:43.695505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:43.695505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:43.752570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ead456-ca16-4b54-8c55-6d1a0bf659a6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:43.756437 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bdc83-bf54-41ef-9da7-229492fc56ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:43.757097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdc83-bf54-41ef-9da7-229492fc56ec service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:43.757383 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdc83-bf54-41ef-9da7-229492fc56ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:43.757662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdc83-bf54-41ef-9da7-229492fc56ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:43.772654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bdc83-bf54-41ef-9da7-229492fc56ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:43.774628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bdc83-bf54-41ef-9da7-229492fc56ec service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:43.786450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1274/2554] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:42:44.064259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73735a-f050-4e84-b3d6-dbb2a94b106e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.065890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ea73735a-f050-4e84-b3d6-dbb2a94b106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:44.079998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73735a-f050-4e84-b3d6-dbb2a94b106e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:44.087521 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:44.088549 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.088900 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.089271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] PATH_INFO: `/v3/projects/c09f808b0aae46e1abbd8055cf207af9` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.090547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09f808b0aae46e1abbd8055cf207af9)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:44.092583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:44.144987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09f808b0aae46e1abbd8055cf207af9', 'request_id': 'req-81097606-cd25-432a-a41c-e43274535233'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:44.151366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09f808b0aae46e1abbd8055cf207af9', 'request_id': 'req-81097606-cd25-432a-a41c-e43274535233'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:44.162345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09f808b0aae46e1abbd8055cf207af9', 'request_id': 'req-81097606-cd25-432a-a41c-e43274535233'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:44.180055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] The token cache is being invalidate because project c09f808b0aae46e1abbd8055cf207af9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:44.180807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:44.180924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81097606-cd25-432a-a41c-e43274535233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:44.182537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1281/2555] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:44 2022] DELETE /identity/v3/projects/c09f808b0aae46e1abbd8055cf207af9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:44.188220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac648dc-f88e-4245-b011-d7e50e5050ba None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.188530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac648dc-f88e-4245-b011-d7e50e5050ba None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.188810 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac648dc-f88e-4245-b011-d7e50e5050ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.208008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5ac648dc-f88e-4245-b011-d7e50e5050ba None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:44.209219 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5ac648dc-f88e-4245-b011-d7e50e5050ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:44.269482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1275/2556] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:44.278978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c550-525d-44f0-8dd1-81b00d05b991 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.279238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c550-525d-44f0-8dd1-81b00d05b991 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.279584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527c550-525d-44f0-8dd1-81b00d05b991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.299560 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a527c550-525d-44f0-8dd1-81b00d05b991 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:44.300759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a527c550-525d-44f0-8dd1-81b00d05b991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:44.309777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1282/2557] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:44.314852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac648dc-f88e-4245-b011-d7e50e5050ba None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.328156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac648dc-f88e-4245-b011-d7e50e5050ba None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:44.333228 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f48b6-9fc3-4865-a534-eadbb9d20ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:44.333891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f48b6-9fc3-4865-a534-eadbb9d20ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.334171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f48b6-9fc3-4865-a534-eadbb9d20ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.334471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f48b6-9fc3-4865-a534-eadbb9d20ff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.335196 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f48b6-9fc3-4865-a534-eadbb9d20ff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:42:44.335535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f48b6-9fc3-4865-a534-eadbb9d20ff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:44.337281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f48b6-9fc3-4865-a534-eadbb9d20ff2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:44.349940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1276/2558] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:42:44.354567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c550-525d-44f0-8dd1-81b00d05b991 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.364555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527c550-525d-44f0-8dd1-81b00d05b991 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:44.371068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7833e9-3576-4a58-aadf-57b1a4f3f6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:44.371756 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7833e9-3576-4a58-aadf-57b1a4f3f6fe admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.372034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7833e9-3576-4a58-aadf-57b1a4f3f6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.372412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7833e9-3576-4a58-aadf-57b1a4f3f6fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.373120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7833e9-3576-4a58-aadf-57b1a4f3f6fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:44.375025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7833e9-3576-4a58-aadf-57b1a4f3f6fe admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:44.399113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:44.408137 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1283/2559] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:44 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:44.413567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f48b6-9fc3-4865-a534-eadbb9d20ff2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.424750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f48b6-9fc3-4865-a534-eadbb9d20ff2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:44.445182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:44.491897 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6faa49-cba9-4767-bc5c-854a95db774a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:44.492610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6faa49-cba9-4767-bc5c-854a95db774a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.492998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6faa49-cba9-4767-bc5c-854a95db774a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.493356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6faa49-cba9-4767-bc5c-854a95db774a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.494251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6faa49-cba9-4767-bc5c-854a95db774a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:44.496523 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6faa49-cba9-4767-bc5c-854a95db774a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:44.518682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1277/2560] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:44 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:42:44.523097 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7833e9-3576-4a58-aadf-57b1a4f3f6fe admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.540203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7833e9-3576-4a58-aadf-57b1a4f3f6fe admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:44.546440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85ef2-475a-46d7-b25a-8b991fb51f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:44.547113 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85ef2-475a-46d7-b25a-8b991fb51f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.547425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85ef2-475a-46d7-b25a-8b991fb51f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.547701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85ef2-475a-46d7-b25a-8b991fb51f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.548433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a85ef2-475a-46d7-b25a-8b991fb51f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:44.550060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a85ef2-475a-46d7-b25a-8b991fb51f40 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:44.562075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1284/2561] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:44.568133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6faa49-cba9-4767-bc5c-854a95db774a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.583517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6faa49-cba9-4767-bc5c-854a95db774a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:44.590033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706f320-ceb4-433e-965c-f7a8298f04e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:44.590734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706f320-ceb4-433e-965c-f7a8298f04e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.591154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706f320-ceb4-433e-965c-f7a8298f04e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.591578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706f320-ceb4-433e-965c-f7a8298f04e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.592319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706f320-ceb4-433e-965c-f7a8298f04e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:44.593975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706f320-ceb4-433e-965c-f7a8298f04e2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:44.604914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1278/2562] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:44.609195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85ef2-475a-46d7-b25a-8b991fb51f40 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.619427 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85ef2-475a-46d7-b25a-8b991fb51f40 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:44.624605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7f657-4086-4b0f-8f1a-473d2b39398b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:44.625526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7f657-4086-4b0f-8f1a-473d2b39398b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.625802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7f657-4086-4b0f-8f1a-473d2b39398b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.626077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7f657-4086-4b0f-8f1a-473d2b39398b admin admin] PATH_INFO: `/v3/projects/316de95f2670430181d64871d7ebf5a9/users/de2dc359adbe4204899b3494c318cfed/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.641614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7f657-4086-4b0f-8f1a-473d2b39398b admin admin] RBAC: Authorizing `identity:create_grant(project_id=316de95f2670430181d64871d7ebf5a9, user_id=de2dc359adbe4204899b3494c318cfed, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:44.644400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7f657-4086-4b0f-8f1a-473d2b39398b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:44.659878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1285/2563] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:44 2022] PUT /identity/v3/projects/316de95f2670430181d64871d7ebf5a9/users/de2dc359adbe4204899b3494c318cfed/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:44.671974 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94f4fb-f9c0-4cd6-9dd5-572d03a4eb9b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.672369 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94f4fb-f9c0-4cd6-9dd5-572d03a4eb9b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.672647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94f4fb-f9c0-4cd6-9dd5-572d03a4eb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.701944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1d94f4fb-f9c0-4cd6-9dd5-572d03a4eb9b None None] MFA Rules not processed for user `de2dc359adbe4204899b3494c318cfed`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:44.703090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1d94f4fb-f9c0-4cd6-9dd5-572d03a4eb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:44.773492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1279/2564] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:44.778350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7f657-4086-4b0f-8f1a-473d2b39398b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.848273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7f657-4086-4b0f-8f1a-473d2b39398b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:44.853148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd00b0c-1f6f-49f6-8895-aedfb2431c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:44.853814 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd00b0c-1f6f-49f6-8895-aedfb2431c63 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.854093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd00b0c-1f6f-49f6-8895-aedfb2431c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.854479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd00b0c-1f6f-49f6-8895-aedfb2431c63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.855183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd00b0c-1f6f-49f6-8895-aedfb2431c63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:44.857492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd00b0c-1f6f-49f6-8895-aedfb2431c63 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:44.886438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1286/2565] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:44 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:44.890781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94f4fb-f9c0-4cd6-9dd5-572d03a4eb9b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.902966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94f4fb-f9c0-4cd6-9dd5-572d03a4eb9b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:44.975702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd00b0c-1f6f-49f6-8895-aedfb2431c63 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:44.977295 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04937c51-8d39-46d0-8eab-90822dcb4a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:44.977432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7cd00b0c-1f6f-49f6-8895-aedfb2431c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:44.978175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04937c51-8d39-46d0-8eab-90822dcb4a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:44.978469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04937c51-8d39-46d0-8eab-90822dcb4a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:44.978761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04937c51-8d39-46d0-8eab-90822dcb4a1e admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:44.979518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04937c51-8d39-46d0-8eab-90822dcb4a1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:44.981469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04937c51-8d39-46d0-8eab-90822dcb4a1e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:45.003785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1280/2566] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:44 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:45.008072 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04937c51-8d39-46d0-8eab-90822dcb4a1e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:45.018002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04937c51-8d39-46d0-8eab-90822dcb4a1e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:45.022648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997ffc3-fc80-4e7a-ba80-dfc3e428cbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:45.023349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997ffc3-fc80-4e7a-ba80-dfc3e428cbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:45.023634 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997ffc3-fc80-4e7a-ba80-dfc3e428cbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:45.023910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997ffc3-fc80-4e7a-ba80-dfc3e428cbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:45.024631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997ffc3-fc80-4e7a-ba80-dfc3e428cbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:45.026232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997ffc3-fc80-4e7a-ba80-dfc3e428cbe5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:45.038285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1281/2567] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:42:45.038475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd00b0c-1f6f-49f6-8895-aedfb2431c63 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:45.040578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997ffc3-fc80-4e7a-ba80-dfc3e428cbe5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:45.041992 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c577c-57c1-41f3-af7c-2b9c1f4c55ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:45.042645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c577c-57c1-41f3-af7c-2b9c1f4c55ed service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:45.042928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c577c-57c1-41f3-af7c-2b9c1f4c55ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:45.043201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c577c-57c1-41f3-af7c-2b9c1f4c55ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:45.045273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5f0c577c-57c1-41f3-af7c-2b9c1f4c55ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:45.050406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997ffc3-fc80-4e7a-ba80-dfc3e428cbe5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:45.055040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8046c-0d3c-462c-a2c1-332febc13284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:45.055948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8046c-0d3c-462c-a2c1-332febc13284 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:45.056227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8046c-0d3c-462c-a2c1-332febc13284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:45.056585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8046c-0d3c-462c-a2c1-332febc13284 admin admin] PATH_INFO: `/v3/projects/65d60147b73b44948588e881668325b0/users/d8383897b2214ca9a3d663c720916cc1/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:45.057441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c577c-57c1-41f3-af7c-2b9c1f4c55ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:45.058923 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c577c-57c1-41f3-af7c-2b9c1f4c55ed service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:45.060669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:45.070605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1287/2568] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:42:45.075062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8046c-0d3c-462c-a2c1-332febc13284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d60147b73b44948588e881668325b0, user_id=d8383897b2214ca9a3d663c720916cc1, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:45.077994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8046c-0d3c-462c-a2c1-332febc13284 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:45.095547 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1282/2569] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:45 2022] PUT /identity/v3/projects/65d60147b73b44948588e881668325b0/users/d8383897b2214ca9a3d663c720916cc1/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:45.099803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c577c-57c1-41f3-af7c-2b9c1f4c55ed service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:45.112455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c577c-57c1-41f3-af7c-2b9c1f4c55ed service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:45.161099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:45.173029 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f2c46-06cb-4056-b51d-ee7b5da7f217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:45.173732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f2c46-06cb-4056-b51d-ee7b5da7f217 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:45.174020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f2c46-06cb-4056-b51d-ee7b5da7f217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:45.174297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f2c46-06cb-4056-b51d-ee7b5da7f217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:45.175040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f2c46-06cb-4056-b51d-ee7b5da7f217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:45.176716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f2c46-06cb-4056-b51d-ee7b5da7f217 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:45.191626 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1288/2570] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:42:45.193618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8046c-0d3c-462c-a2c1-332febc13284 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:45.205286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8046c-0d3c-462c-a2c1-332febc13284 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:45.211010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3316352-3db2-4b10-83ec-a6a6fe16f4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:45.212123 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3316352-3db2-4b10-83ec-a6a6fe16f4e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:45.212400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3316352-3db2-4b10-83ec-a6a6fe16f4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:45.212676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3316352-3db2-4b10-83ec-a6a6fe16f4e4 admin admin] PATH_INFO: `/v3/projects/65d60147b73b44948588e881668325b0/users/d8383897b2214ca9a3d663c720916cc1/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:45.214984 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3316352-3db2-4b10-83ec-a6a6fe16f4e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d60147b73b44948588e881668325b0, user_id=d8383897b2214ca9a3d663c720916cc1, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:45.217931 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3316352-3db2-4b10-83ec-a6a6fe16f4e4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:45.230296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1283/2571] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:45 2022] PUT /identity/v3/projects/65d60147b73b44948588e881668325b0/users/d8383897b2214ca9a3d663c720916cc1/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:45.240932 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200f074-5d95-4937-ba11-700eb97d1c1c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:45.241219 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200f074-5d95-4937-ba11-700eb97d1c1c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:45.241495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200f074-5d95-4937-ba11-700eb97d1c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:45.271862 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4200f074-5d95-4937-ba11-700eb97d1c1c None None] MFA Rules not processed for user `d8383897b2214ca9a3d663c720916cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:45.273094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4200f074-5d95-4937-ba11-700eb97d1c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:45.350460 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1289/2572] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:45.362154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3316352-3db2-4b10-83ec-a6a6fe16f4e4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:45.364055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e3316352-3db2-4b10-83ec-a6a6fe16f4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:45.427929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3316352-3db2-4b10-83ec-a6a6fe16f4e4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:45.431793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22ff86-b5f8-4f08-82cb-d8951274dfdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:45.432475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22ff86-b5f8-4f08-82cb-d8951274dfdb service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:45.432758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22ff86-b5f8-4f08-82cb-d8951274dfdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:45.433042 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22ff86-b5f8-4f08-82cb-d8951274dfdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:45.490621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22ff86-b5f8-4f08-82cb-d8951274dfdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:45.492157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22ff86-b5f8-4f08-82cb-d8951274dfdb service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:45.501543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1284/2573] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:48.117414 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200f074-5d95-4937-ba11-700eb97d1c1c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:48.197707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200f074-5d95-4937-ba11-700eb97d1c1c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:48.204119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:48.205108 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:48.205471 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:48.205807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] PATH_INFO: `/v3/users/de2dc359adbe4204899b3494c318cfed` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:48.207421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2dc359adbe4204899b3494c318cfed)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:48.209516 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:48.273954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2dc359adbe4204899b3494c318cfed'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.275869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2dc359adbe4204899b3494c318cfed'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.287865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2dc359adbe4204899b3494c318cfed'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.308381 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2dc359adbe4204899b3494c318cfed'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.309376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2dc359adbe4204899b3494c318cfed'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.312804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:48.329343 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1290/2574] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:48 2022] DELETE /identity/v3/users/de2dc359adbe4204899b3494c318cfed => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:48.333948 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720c679-2615-4b59-9e8e-19096522452a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:48.334457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720c679-2615-4b59-9e8e-19096522452a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:48.334563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720c679-2615-4b59-9e8e-19096522452a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:48.351759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1720c679-2615-4b59-9e8e-19096522452a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:48.352982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1720c679-2615-4b59-9e8e-19096522452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:48.423283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1285/2575] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:48.434178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:48.435162 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:48.500510 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec02d0-aa39-4ced-a8ab-d1cad4d8c333 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:48.508892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e841ab-f00f-469b-94fa-c20316197ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:48.509852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e841ab-f00f-469b-94fa-c20316197ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:48.510240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e841ab-f00f-469b-94fa-c20316197ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:48.511047 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e841ab-f00f-469b-94fa-c20316197ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:48.521716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e841ab-f00f-469b-94fa-c20316197ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:48.523388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e841ab-f00f-469b-94fa-c20316197ab6 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:48.535042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1291/2576] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:48.858995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720c679-2615-4b59-9e8e-19096522452a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:48.861391 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1720c679-2615-4b59-9e8e-19096522452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:48.885890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720c679-2615-4b59-9e8e-19096522452a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:48.895790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:48.896933 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:48.897263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:48.897609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] PATH_INFO: `/v3/projects/316de95f2670430181d64871d7ebf5a9` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:48.899543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=316de95f2670430181d64871d7ebf5a9)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:48.902707 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:48.933923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:48.961142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316de95f2670430181d64871d7ebf5a9', 'request_id': 'req-1ca50722-6e94-45c8-877e-41c549babd84'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.970063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316de95f2670430181d64871d7ebf5a9', 'request_id': 'req-1ca50722-6e94-45c8-877e-41c549babd84'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.976741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316de95f2670430181d64871d7ebf5a9', 'request_id': 'req-1ca50722-6e94-45c8-877e-41c549babd84'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.987502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] The token cache is being invalidate because project 316de95f2670430181d64871d7ebf5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:48.987714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.988356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:48.989914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1286/2577] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:48 2022] DELETE /identity/v3/projects/316de95f2670430181d64871d7ebf5a9 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:48.994426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e841ab-f00f-469b-94fa-c20316197ab6 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:49.007094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e841ab-f00f-469b-94fa-c20316197ab6 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:49.062805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:49.063724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:49.064008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:49.064287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] PATH_INFO: `/v3/users/d8383897b2214ca9a3d663c720916cc1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:49.065470 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8383897b2214ca9a3d663c720916cc1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:49.067436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:49.120347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8383897b2214ca9a3d663c720916cc1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.121066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8383897b2214ca9a3d663c720916cc1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.131203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8383897b2214ca9a3d663c720916cc1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.145293 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8383897b2214ca9a3d663c720916cc1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.146058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ec1bd641-2dd4-44ef-9b64-b29eeb7b88d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8383897b2214ca9a3d663c720916cc1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.160149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1292/2578] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:48 2022] DELETE /identity/v3/users/d8383897b2214ca9a3d663c720916cc1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:49.440560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:49.442532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:49.457120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca50722-6e94-45c8-877e-41c549babd84 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:49.527007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:49.527831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:49.528096 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:49.528449 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] PATH_INFO: `/v3/projects/65d60147b73b44948588e881668325b0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:49.530399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d60147b73b44948588e881668325b0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:49.532393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:49.585010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d60147b73b44948588e881668325b0', 'request_id': 'req-880d944e-80dd-4f3f-935e-5875d9540135'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.595588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d60147b73b44948588e881668325b0', 'request_id': 'req-880d944e-80dd-4f3f-935e-5875d9540135'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.603023 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d60147b73b44948588e881668325b0', 'request_id': 'req-880d944e-80dd-4f3f-935e-5875d9540135'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.615137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] The token cache is being invalidate because project 65d60147b73b44948588e881668325b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:49.615458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.616153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-880d944e-80dd-4f3f-935e-5875d9540135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:49.617763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1287/2579] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:49 2022] DELETE /identity/v3/projects/65d60147b73b44948588e881668325b0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:49.622938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00575d2d-c85f-4292-bd11-cbf98859962a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:49.623402 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00575d2d-c85f-4292-bd11-cbf98859962a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:49.623679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00575d2d-c85f-4292-bd11-cbf98859962a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:49.642973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-00575d2d-c85f-4292-bd11-cbf98859962a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:49.644235 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-00575d2d-c85f-4292-bd11-cbf98859962a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:49.654020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:49.710664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1293/2580] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:49.719559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b90fc9-2749-4903-9217-3de7a980696d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:49.719853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b90fc9-2749-4903-9217-3de7a980696d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:49.720138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b90fc9-2749-4903-9217-3de7a980696d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:49.738786 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-06b90fc9-2749-4903-9217-3de7a980696d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:49.740022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-06b90fc9-2749-4903-9217-3de7a980696d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:49.752355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1288/2581] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:49.756782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00575d2d-c85f-4292-bd11-cbf98859962a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:49.766734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00575d2d-c85f-4292-bd11-cbf98859962a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:49.771254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d8589-0aee-4c2e-81ec-397bd5c97a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:49.771944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d8589-0aee-4c2e-81ec-397bd5c97a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:49.772657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d8589-0aee-4c2e-81ec-397bd5c97a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:49.772657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d8589-0aee-4c2e-81ec-397bd5c97a83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:49.773215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d8589-0aee-4c2e-81ec-397bd5c97a83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:42:49.773524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d8589-0aee-4c2e-81ec-397bd5c97a83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:49.775132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d8589-0aee-4c2e-81ec-397bd5c97a83 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:49.787253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1294/2582] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:42:49.791953 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b90fc9-2749-4903-9217-3de7a980696d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:49.802077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b90fc9-2749-4903-9217-3de7a980696d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:49.807515 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1c068-7848-4e2c-92b6-88f22855d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:49.808183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1c068-7848-4e2c-92b6-88f22855d09f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:49.808463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1c068-7848-4e2c-92b6-88f22855d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:49.808856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1c068-7848-4e2c-92b6-88f22855d09f admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:49.809589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1c068-7848-4e2c-92b6-88f22855d09f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:49.811582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1c068-7848-4e2c-92b6-88f22855d09f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:49.843180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1289/2583] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:49 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:49.847737 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d8589-0aee-4c2e-81ec-397bd5c97a83 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:49.866114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d8589-0aee-4c2e-81ec-397bd5c97a83 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:49.934694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860240be-cdc4-4cdb-a420-d9c29224b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:49.935420 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860240be-cdc4-4cdb-a420-d9c29224b931 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:49.935707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860240be-cdc4-4cdb-a420-d9c29224b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:49.935986 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860240be-cdc4-4cdb-a420-d9c29224b931 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:49.936691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860240be-cdc4-4cdb-a420-d9c29224b931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:49.938685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860240be-cdc4-4cdb-a420-d9c29224b931 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:49.960876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1295/2584] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:49 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:49.965205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1c068-7848-4e2c-92b6-88f22855d09f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:49.980925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1c068-7848-4e2c-92b6-88f22855d09f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:49.986094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f718e6f-b521-4361-8c4d-345c3ceac778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:49.986788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f718e6f-b521-4361-8c4d-345c3ceac778 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:49.987071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f718e6f-b521-4361-8c4d-345c3ceac778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:49.987380 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f718e6f-b521-4361-8c4d-345c3ceac778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:49.988115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f718e6f-b521-4361-8c4d-345c3ceac778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:49.989753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f718e6f-b521-4361-8c4d-345c3ceac778 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:49.996496 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:50.007853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1290/2585] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:50.009417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860240be-cdc4-4cdb-a420-d9c29224b931 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:50.022144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860240be-cdc4-4cdb-a420-d9c29224b931 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:50.027433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7a8b4-2acf-4c02-af08-b662512c7c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:50.028112 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7a8b4-2acf-4c02-af08-b662512c7c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:50.028755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7a8b4-2acf-4c02-af08-b662512c7c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:50.029041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7a8b4-2acf-4c02-af08-b662512c7c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:50.029779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7a8b4-2acf-4c02-af08-b662512c7c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:50.031449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7a8b4-2acf-4c02-af08-b662512c7c2c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:50.042666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1296/2586] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:50.046961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f718e6f-b521-4361-8c4d-345c3ceac778 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:50.057322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f718e6f-b521-4361-8c4d-345c3ceac778 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:50.062078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179088d-1eb4-4e9d-bdc1-043e9aa68a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:50.062966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179088d-1eb4-4e9d-bdc1-043e9aa68a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:50.063247 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179088d-1eb4-4e9d-bdc1-043e9aa68a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:50.063561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179088d-1eb4-4e9d-bdc1-043e9aa68a81 admin admin] PATH_INFO: `/v3/projects/a713cd9e5960426a9378c54dd45e4997/users/2d59b611ec3145d0ba5dc08999f6bdaa/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:50.078417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179088d-1eb4-4e9d-bdc1-043e9aa68a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a713cd9e5960426a9378c54dd45e4997, user_id=2d59b611ec3145d0ba5dc08999f6bdaa, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:50.081148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179088d-1eb4-4e9d-bdc1-043e9aa68a81 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:50.098230 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1291/2587] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:50 2022] PUT /identity/v3/projects/a713cd9e5960426a9378c54dd45e4997/users/2d59b611ec3145d0ba5dc08999f6bdaa/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:50.111613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de96c64-35c4-4389-882e-7e01fb5a16af None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:50.112366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de96c64-35c4-4389-882e-7e01fb5a16af None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:50.112366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de96c64-35c4-4389-882e-7e01fb5a16af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:50.142079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2de96c64-35c4-4389-882e-7e01fb5a16af None None] MFA Rules not processed for user `2d59b611ec3145d0ba5dc08999f6bdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:50.143423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2de96c64-35c4-4389-882e-7e01fb5a16af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:50.214351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1297/2588] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:50.225448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179088d-1eb4-4e9d-bdc1-043e9aa68a81 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:50.227334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8179088d-1eb4-4e9d-bdc1-043e9aa68a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:50.290198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179088d-1eb4-4e9d-bdc1-043e9aa68a81 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:50.294848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856669f9-882c-4aff-9c07-e6a84a20b389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:50.295642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856669f9-882c-4aff-9c07-e6a84a20b389 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:50.296026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856669f9-882c-4aff-9c07-e6a84a20b389 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:50.296421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856669f9-882c-4aff-9c07-e6a84a20b389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:50.306568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856669f9-882c-4aff-9c07-e6a84a20b389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:50.308235 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856669f9-882c-4aff-9c07-e6a84a20b389 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:50.318164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1292/2589] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:51.743595 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40d03a-e75d-4678-ba12-64199c9f94d4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:51.744531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40d03a-e75d-4678-ba12-64199c9f94d4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:51.744753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40d03a-e75d-4678-ba12-64199c9f94d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:51.767506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ca40d03a-e75d-4678-ba12-64199c9f94d4 None None] MFA Rules not processed for user `2d59b611ec3145d0ba5dc08999f6bdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:51.769348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ca40d03a-e75d-4678-ba12-64199c9f94d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:51.780536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1298/2590] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:52.032431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856669f9-882c-4aff-9c07-e6a84a20b389 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:52.033917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-856669f9-882c-4aff-9c07-e6a84a20b389 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:52.119030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856669f9-882c-4aff-9c07-e6a84a20b389 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:52.123852 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:52.124732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:52.125014 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:52.125287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] PATH_INFO: `/v3/users/4f509b8a8e3c4bc3a5143d44c078d939` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:52.126685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f509b8a8e3c4bc3a5143d44c078d939)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:52.128660 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:52.179097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f509b8a8e3c4bc3a5143d44c078d939'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.179781 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f509b8a8e3c4bc3a5143d44c078d939'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.189328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f509b8a8e3c4bc3a5143d44c078d939'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.190057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f509b8a8e3c4bc3a5143d44c078d939'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.203064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f509b8a8e3c4bc3a5143d44c078d939'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.210561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:52.217280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1293/2591] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:52 2022] DELETE /identity/v3/users/4f509b8a8e3c4bc3a5143d44c078d939 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:52.411815 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40d03a-e75d-4678-ba12-64199c9f94d4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:52.419966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:52.426713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40d03a-e75d-4678-ba12-64199c9f94d4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:52.489853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:52.491109 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:52.491566 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:52.491901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] PATH_INFO: `/v3/projects/6c04290d87154e48a3bf77a828b563ba` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:52.493324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c04290d87154e48a3bf77a828b563ba)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:52.495273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:52.549836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c04290d87154e48a3bf77a828b563ba', 'request_id': 'req-7b9bc82a-9455-4995-9575-0c17dd0ffd07'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.556530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c04290d87154e48a3bf77a828b563ba', 'request_id': 'req-7b9bc82a-9455-4995-9575-0c17dd0ffd07'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.564241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c04290d87154e48a3bf77a828b563ba', 'request_id': 'req-7b9bc82a-9455-4995-9575-0c17dd0ffd07'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.577046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] The token cache is being invalidate because project 6c04290d87154e48a3bf77a828b563ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:52.577418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.578032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.579755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1299/2592] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:52 2022] DELETE /identity/v3/projects/6c04290d87154e48a3bf77a828b563ba => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:52.583875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:52.585401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:52.599943 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab12f2-2fe7-4c79-854b-d0569832e8be admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:52.679118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:52.680180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:52.680539 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:52.680876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] PATH_INFO: `/v3/users/5c014213af9f44869af94ce258d265e4` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:52.682248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c014213af9f44869af94ce258d265e4)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:52.684245 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:52.740545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c014213af9f44869af94ce258d265e4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.741191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c014213af9f44869af94ce258d265e4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.750968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c014213af9f44869af94ce258d265e4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.751906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c014213af9f44869af94ce258d265e4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.764372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-c74bda22-eb14-4f46-93fa-917b09b68dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c014213af9f44869af94ce258d265e4'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:52.778134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1294/2593] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:42:52 2022] DELETE /identity/v3/users/5c014213af9f44869af94ce258d265e4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:52.967751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:52.969479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:52.985450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bc82a-9455-4995-9575-0c17dd0ffd07 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:53.048091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:53.048952 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.049231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.049502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] PATH_INFO: `/v3/projects/49622e8284454d679cacdd88b2f2344a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.050731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=49622e8284454d679cacdd88b2f2344a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:53.052732 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:53.092664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:53.104315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49622e8284454d679cacdd88b2f2344a', 'request_id': 'req-49aa0878-d3e6-4f57-a698-d3d77494e8fd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:53.109977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49622e8284454d679cacdd88b2f2344a', 'request_id': 'req-49aa0878-d3e6-4f57-a698-d3d77494e8fd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:53.116240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49622e8284454d679cacdd88b2f2344a', 'request_id': 'req-49aa0878-d3e6-4f57-a698-d3d77494e8fd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:53.130296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] The token cache is being invalidate because project 49622e8284454d679cacdd88b2f2344a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:42:53.130582 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:53.131224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-49aa0878-d3e6-4f57-a698-d3d77494e8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:42:53.132797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1300/2594] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:42:52 2022] DELETE /identity/v3/projects/49622e8284454d679cacdd88b2f2344a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:53.138239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24b7c-e0d1-42f0-9ba7-b04d094196c9 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.138805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24b7c-e0d1-42f0-9ba7-b04d094196c9 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.138805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24b7c-e0d1-42f0-9ba7-b04d094196c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.157158 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-adb24b7c-e0d1-42f0-9ba7-b04d094196c9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:53.158359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-adb24b7c-e0d1-42f0-9ba7-b04d094196c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:53.223364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1295/2595] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:42:53.231885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed76cb5-9b89-4620-9c0a-9c6a42afc9d8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.232181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed76cb5-9b89-4620-9c0a-9c6a42afc9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.232477 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed76cb5-9b89-4620-9c0a-9c6a42afc9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.251769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-eed76cb5-9b89-4620-9c0a-9c6a42afc9d8 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:53.252979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eed76cb5-9b89-4620-9c0a-9c6a42afc9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:53.261357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1301/2596] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:53.265720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb24b7c-e0d1-42f0-9ba7-b04d094196c9 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:53.276198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb24b7c-e0d1-42f0-9ba7-b04d094196c9 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:53.283163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82c9d4-8785-4ef4-b3df-10e6abf84fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:53.284288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82c9d4-8785-4ef4-b3df-10e6abf84fed admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.284698 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82c9d4-8785-4ef4-b3df-10e6abf84fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.285240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82c9d4-8785-4ef4-b3df-10e6abf84fed admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.286015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82c9d4-8785-4ef4-b3df-10e6abf84fed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:42:53.286335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82c9d4-8785-4ef4-b3df-10e6abf84fed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:53.288613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82c9d4-8785-4ef4-b3df-10e6abf84fed admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:53.303328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1296/2597] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:42:53.307505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed76cb5-9b89-4620-9c0a-9c6a42afc9d8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:53.320328 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed76cb5-9b89-4620-9c0a-9c6a42afc9d8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:53.328580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff58c2-bd52-4dc8-92a7-3cb32b1106e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:53.329340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff58c2-bd52-4dc8-92a7-3cb32b1106e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.329627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff58c2-bd52-4dc8-92a7-3cb32b1106e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.329914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff58c2-bd52-4dc8-92a7-3cb32b1106e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.330950 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff58c2-bd52-4dc8-92a7-3cb32b1106e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:53.333358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff58c2-bd52-4dc8-92a7-3cb32b1106e5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:53.370353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1302/2598] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:53 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:42:53.375143 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82c9d4-8785-4ef4-b3df-10e6abf84fed admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:53.384902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:53.390364 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82c9d4-8785-4ef4-b3df-10e6abf84fed admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:53.486510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad8f21-073d-4f39-96df-014274c564a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:53.487442 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad8f21-073d-4f39-96df-014274c564a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.487752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad8f21-073d-4f39-96df-014274c564a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.488026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad8f21-073d-4f39-96df-014274c564a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.488728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad8f21-073d-4f39-96df-014274c564a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:53.490676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad8f21-073d-4f39-96df-014274c564a5 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:53.514576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1297/2599] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:53 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:53.519014 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff58c2-bd52-4dc8-92a7-3cb32b1106e5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:53.531250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff58c2-bd52-4dc8-92a7-3cb32b1106e5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:53.535753 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031964d4-fe0b-46b4-8dc6-7d32a4702256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:53.536430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031964d4-fe0b-46b4-8dc6-7d32a4702256 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.536716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031964d4-fe0b-46b4-8dc6-7d32a4702256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.536996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031964d4-fe0b-46b4-8dc6-7d32a4702256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.537728 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031964d4-fe0b-46b4-8dc6-7d32a4702256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:53.539484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031964d4-fe0b-46b4-8dc6-7d32a4702256 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:53.550729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1303/2600] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:53.554912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad8f21-073d-4f39-96df-014274c564a5 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:53.565681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad8f21-073d-4f39-96df-014274c564a5 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:53.570265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3421c01-a1b7-4a5c-8efa-fe45c177c749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:53.570921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3421c01-a1b7-4a5c-8efa-fe45c177c749 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.571201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3421c01-a1b7-4a5c-8efa-fe45c177c749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.571503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3421c01-a1b7-4a5c-8efa-fe45c177c749 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.572222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3421c01-a1b7-4a5c-8efa-fe45c177c749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:53.573818 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3421c01-a1b7-4a5c-8efa-fe45c177c749 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:53.585473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1298/2601] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:53.590330 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031964d4-fe0b-46b4-8dc6-7d32a4702256 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:53.602576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031964d4-fe0b-46b4-8dc6-7d32a4702256 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:53.607704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7e495-16f0-466f-89ed-94f1401b85e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:53.608601 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7e495-16f0-466f-89ed-94f1401b85e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.608884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7e495-16f0-466f-89ed-94f1401b85e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.609158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7e495-16f0-466f-89ed-94f1401b85e0 admin admin] PATH_INFO: `/v3/projects/01fccbd11a4b4e859bec1a56f42315b2/users/d42b42214d0d46bbb9fe90aed7f5e382/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.624034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7e495-16f0-466f-89ed-94f1401b85e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01fccbd11a4b4e859bec1a56f42315b2, user_id=d42b42214d0d46bbb9fe90aed7f5e382, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:53.626712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7e495-16f0-466f-89ed-94f1401b85e0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:53.641111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1304/2602] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:53 2022] PUT /identity/v3/projects/01fccbd11a4b4e859bec1a56f42315b2/users/d42b42214d0d46bbb9fe90aed7f5e382/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:42:53.646278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d132a4-d5d1-4144-b598-e71b2869a474 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.646564 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d132a4-d5d1-4144-b598-e71b2869a474 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.646994 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d132a4-d5d1-4144-b598-e71b2869a474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.664618 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-68d132a4-d5d1-4144-b598-e71b2869a474 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:53.665812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-68d132a4-d5d1-4144-b598-e71b2869a474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:53.726176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1299/2603] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:53.735930 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7e495-16f0-466f-89ed-94f1401b85e0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:53.738166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7fd7e495-16f0-466f-89ed-94f1401b85e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:53.802204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7e495-16f0-466f-89ed-94f1401b85e0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:53.806076 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa3e38-5601-4d60-9c21-19ebd78c0e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:53.806828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa3e38-5601-4d60-9c21-19ebd78c0e4c service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:53.807115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa3e38-5601-4d60-9c21-19ebd78c0e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:53.807415 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa3e38-5601-4d60-9c21-19ebd78c0e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:53.817012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa3e38-5601-4d60-9c21-19ebd78c0e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:53.818469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa3e38-5601-4d60-9c21-19ebd78c0e4c service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:53.828307 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1305/2604] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:42:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:42:58.444555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76881154-9185-4cc3-bbba-959771ff8608 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:58.445458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76881154-9185-4cc3-bbba-959771ff8608 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:58.445993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76881154-9185-4cc3-bbba-959771ff8608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:58.484761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-76881154-9185-4cc3-bbba-959771ff8608 None None] MFA Rules not processed for user `d42b42214d0d46bbb9fe90aed7f5e382`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:42:58.486319 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-76881154-9185-4cc3-bbba-959771ff8608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:42:58.580708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1300/2605] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:42:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:42:58.585152 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa3e38-5601-4d60-9c21-19ebd78c0e4c service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:58.599058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa3e38-5601-4d60-9c21-19ebd78c0e4c service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:58.622249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ac5b3-dfd3-4c4d-aa1d-f7a513d6c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:58.623577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ac5b3-dfd3-4c4d-aa1d-f7a513d6c39b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:58.624058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ac5b3-dfd3-4c4d-aa1d-f7a513d6c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:58.624537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ac5b3-dfd3-4c4d-aa1d-f7a513d6c39b admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:58.625695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ac5b3-dfd3-4c4d-aa1d-f7a513d6c39b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:58.628007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ac5b3-dfd3-4c4d-aa1d-f7a513d6c39b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:58.685654 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1306/2606] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:42:58 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:42:58.687166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76881154-9185-4cc3-bbba-959771ff8608 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:58.708375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76881154-9185-4cc3-bbba-959771ff8608 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:58.737172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:58.824351 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857c0e-c39d-4e23-a0ed-07244bf1098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:58.825073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857c0e-c39d-4e23-a0ed-07244bf1098a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:58.825499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857c0e-c39d-4e23-a0ed-07244bf1098a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:58.826204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a857c0e-c39d-4e23-a0ed-07244bf1098a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:58.826363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a857c0e-c39d-4e23-a0ed-07244bf1098a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:58.828542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a857c0e-c39d-4e23-a0ed-07244bf1098a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:58.863556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1301/2607] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:42:58 2022] POST /identity/v3/users => generated 296 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:42:58.867397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ac5b3-dfd3-4c4d-aa1d-f7a513d6c39b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:58.883104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ac5b3-dfd3-4c4d-aa1d-f7a513d6c39b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:58.890681 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f4fed-0f32-4e5a-ac1a-c1d9edce5205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:58.891413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f4fed-0f32-4e5a-ac1a-c1d9edce5205 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:58.891748 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f4fed-0f32-4e5a-ac1a-c1d9edce5205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:58.892027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f4fed-0f32-4e5a-ac1a-c1d9edce5205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:58.893334 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f4fed-0f32-4e5a-ac1a-c1d9edce5205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:58.894995 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f4fed-0f32-4e5a-ac1a-c1d9edce5205 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:58.947558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1307/2608] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:58.948029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857c0e-c39d-4e23-a0ed-07244bf1098a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:58.961708 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a857c0e-c39d-4e23-a0ed-07244bf1098a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:58.967893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827d07c-e7da-4768-81b0-c7c034f38246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:58.968962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827d07c-e7da-4768-81b0-c7c034f38246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:58.969249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827d07c-e7da-4768-81b0-c7c034f38246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:58.969536 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827d07c-e7da-4768-81b0-c7c034f38246 admin admin] PATH_INFO: `/v3/projects/ad1140025dba4bc184223e702927ad88/users/0a41036331c84530833af5c08cac988e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:58.991287 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827d07c-e7da-4768-81b0-c7c034f38246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1140025dba4bc184223e702927ad88, user_id=0a41036331c84530833af5c08cac988e, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:58.994459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a827d07c-e7da-4768-81b0-c7c034f38246 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:59.018156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1302/2609] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:58 2022] PUT /identity/v3/projects/ad1140025dba4bc184223e702927ad88/users/0a41036331c84530833af5c08cac988e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:42:59.021366 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f4fed-0f32-4e5a-ac1a-c1d9edce5205 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:59.034202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f4fed-0f32-4e5a-ac1a-c1d9edce5205 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:59.068158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:42:59.121004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f63517-3564-4417-906f-1623f4a74ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:59.121708 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f63517-3564-4417-906f-1623f4a74ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:59.122031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f63517-3564-4417-906f-1623f4a74ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:59.122511 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f63517-3564-4417-906f-1623f4a74ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:59.123272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f63517-3564-4417-906f-1623f4a74ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:59.124971 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f63517-3564-4417-906f-1623f4a74ee2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:59.144703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1308/2610] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:42:59.148633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827d07c-e7da-4768-81b0-c7c034f38246 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:42:59.176831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a827d07c-e7da-4768-81b0-c7c034f38246 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:42:59.181322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b294c4a-3d19-4f1e-a4a9-4355e582dded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:42:59.182328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b294c4a-3d19-4f1e-a4a9-4355e582dded admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:42:59.182685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b294c4a-3d19-4f1e-a4a9-4355e582dded admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:42:59.183038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b294c4a-3d19-4f1e-a4a9-4355e582dded admin admin] PATH_INFO: `/v3/projects/ad1140025dba4bc184223e702927ad88/users/0a41036331c84530833af5c08cac988e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:42:59.190583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b294c4a-3d19-4f1e-a4a9-4355e582dded admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1140025dba4bc184223e702927ad88, user_id=0a41036331c84530833af5c08cac988e, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:42:59.194298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b294c4a-3d19-4f1e-a4a9-4355e582dded admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:42:59.208043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1303/2611] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:42:59 2022] PUT /identity/v3/projects/ad1140025dba4bc184223e702927ad88/users/0a41036331c84530833af5c08cac988e/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:43:04.557324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe3bf-6079-4c6b-9777-cdb91ba9c06a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:04.557791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe3bf-6079-4c6b-9777-cdb91ba9c06a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:04.557897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fe3bf-6079-4c6b-9777-cdb91ba9c06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:04.592203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e72fe3bf-6079-4c6b-9777-cdb91ba9c06a None None] MFA Rules not processed for user `0a41036331c84530833af5c08cac988e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:04.593506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e72fe3bf-6079-4c6b-9777-cdb91ba9c06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:04.678323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1309/2612] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:43:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:04.687775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b294c4a-3d19-4f1e-a4a9-4355e582dded admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:04.689765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-3b294c4a-3d19-4f1e-a4a9-4355e582dded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:04.794550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b294c4a-3d19-4f1e-a4a9-4355e582dded admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:04.800048 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32180b-ad45-4145-87a0-5dc849bbc11a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:04.800783 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32180b-ad45-4145-87a0-5dc849bbc11a service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:04.800967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32180b-ad45-4145-87a0-5dc849bbc11a service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:04.801240 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32180b-ad45-4145-87a0-5dc849bbc11a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:04.874067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32180b-ad45-4145-87a0-5dc849bbc11a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:04.875921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32180b-ad45-4145-87a0-5dc849bbc11a service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:04.889183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1304/2613] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:43:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:43:10.169038 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c1b05-28b5-40ab-a64d-a1198cea9d47 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:10.169536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c1b05-28b5-40ab-a64d-a1198cea9d47 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:10.169650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c1b05-28b5-40ab-a64d-a1198cea9d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:10.187661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bc4c1b05-28b5-40ab-a64d-a1198cea9d47 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:10.188884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bc4c1b05-28b5-40ab-a64d-a1198cea9d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:10.245378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1310/2614] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:43:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:10.295596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32180b-ad45-4145-87a0-5dc849bbc11a service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:10.297222 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-cd32180b-ad45-4145-87a0-5dc849bbc11a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:10.302999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:43:10.374264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32180b-ad45-4145-87a0-5dc849bbc11a service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:10.380456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc7b95-e97a-4dae-9068-b81074fa9390 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:10.381120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc7b95-e97a-4dae-9068-b81074fa9390 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:10.381402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc7b95-e97a-4dae-9068-b81074fa9390 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:10.381671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc7b95-e97a-4dae-9068-b81074fa9390 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:10.390979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc7b95-e97a-4dae-9068-b81074fa9390 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:10.392644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc7b95-e97a-4dae-9068-b81074fa9390 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:10.402691 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1305/2615] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:43:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:43:10.940168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5ce01-6f8d-44d9-b472-38b7b9a6d373 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:10.940532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5ce01-6f8d-44d9-b472-38b7b9a6d373 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:10.940880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5ce01-6f8d-44d9-b472-38b7b9a6d373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:10.959345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ecd5ce01-6f8d-44d9-b472-38b7b9a6d373 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:10.960646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ecd5ce01-6f8d-44d9-b472-38b7b9a6d373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:10.970041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1311/2616] 10.222.0.42 () {58 vars in 967 bytes} [Sat Jul 2 03:43:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:10.985430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc7b95-e97a-4dae-9068-b81074fa9390 service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:10.998476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc7b95-e97a-4dae-9068-b81074fa9390 service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:11.004960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7203f90-1d8a-4303-8498-cf5ea19ba2d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:11.005788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7203f90-1d8a-4303-8498-cf5ea19ba2d8 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:11.006173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7203f90-1d8a-4303-8498-cf5ea19ba2d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:11.006535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7203f90-1d8a-4303-8498-cf5ea19ba2d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:11.016033 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7203f90-1d8a-4303-8498-cf5ea19ba2d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:11.017628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7203f90-1d8a-4303-8498-cf5ea19ba2d8 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:11.025742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1306/2617] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:43:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:43:11.245255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a121b-9433-403b-811a-c3b626e840f1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:11.245778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a121b-9433-403b-811a-c3b626e840f1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:11.245958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a121b-9433-403b-811a-c3b626e840f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:11.265093 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-871a121b-9433-403b-811a-c3b626e840f1 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:11.266423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-871a121b-9433-403b-811a-c3b626e840f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:11.275591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1312/2618] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:43:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:11.288242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7203f90-1d8a-4303-8498-cf5ea19ba2d8 service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:11.299227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7203f90-1d8a-4303-8498-cf5ea19ba2d8 service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:11.304184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa22181-c7dd-4922-89d0-794a208de28e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:11.304828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa22181-c7dd-4922-89d0-794a208de28e service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:11.305130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa22181-c7dd-4922-89d0-794a208de28e service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:11.305417 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa22181-c7dd-4922-89d0-794a208de28e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:11.313812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa22181-c7dd-4922-89d0-794a208de28e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:11.315445 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa22181-c7dd-4922-89d0-794a208de28e service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:11.324676 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1307/2619] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:43:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 02 03:43:11.653940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c61f-85a9-4432-8873-919cb00778a3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:11.654311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c61f-85a9-4432-8873-919cb00778a3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:11.654311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322c61f-85a9-4432-8873-919cb00778a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:11.677627 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7322c61f-85a9-4432-8873-919cb00778a3 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:11.678844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7322c61f-85a9-4432-8873-919cb00778a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:11.686901 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1313/2620] 10.222.0.42 () {58 vars in 967 bytes} [Sat Jul 2 03:43:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:11.700386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa22181-c7dd-4922-89d0-794a208de28e service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:11.715193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa22181-c7dd-4922-89d0-794a208de28e service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:11.721006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d7a45-852a-438e-a9f0-98cffd39274f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:11.721650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d7a45-852a-438e-a9f0-98cffd39274f service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:11.721929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d7a45-852a-438e-a9f0-98cffd39274f service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:11.722211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d7a45-852a-438e-a9f0-98cffd39274f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:11.735041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d7a45-852a-438e-a9f0-98cffd39274f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:11.736681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d7a45-852a-438e-a9f0-98cffd39274f service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:11.745987 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1308/2621] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:43:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:43:13.872621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322c61f-85a9-4432-8873-919cb00778a3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:13.874602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7322c61f-85a9-4432-8873-919cb00778a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:13.934977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:43:13.975429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322c61f-85a9-4432-8873-919cb00778a3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:13.980604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:13.981867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:13.982225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:13.982740 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] PATH_INFO: `/v3/users/2d59b611ec3145d0ba5dc08999f6bdaa` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:13.984571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d59b611ec3145d0ba5dc08999f6bdaa)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:13.987973 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:14.054807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d59b611ec3145d0ba5dc08999f6bdaa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.055558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d59b611ec3145d0ba5dc08999f6bdaa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.068661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d59b611ec3145d0ba5dc08999f6bdaa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.082716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d59b611ec3145d0ba5dc08999f6bdaa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.083579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d59b611ec3145d0ba5dc08999f6bdaa'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.097210 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1314/2622] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:43:13 2022] DELETE /identity/v3/users/2d59b611ec3145d0ba5dc08999f6bdaa => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:43:14.101895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b97b3-f5ce-4eb2-8d97-096d5069acdc None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:14.102376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b97b3-f5ce-4eb2-8d97-096d5069acdc None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:14.102472 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b97b3-f5ce-4eb2-8d97-096d5069acdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:14.130801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-066b97b3-f5ce-4eb2-8d97-096d5069acdc None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:14.132135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-066b97b3-f5ce-4eb2-8d97-096d5069acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:14.194667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1309/2623] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:14.204483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:14.206081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:14.274168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53396ee-4e53-49f1-ba39-1c3f07363d83 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:14.278175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59280fc2-e68b-4047-b1f6-edd838aaceb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:14.280128 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59280fc2-e68b-4047-b1f6-edd838aaceb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:14.281175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59280fc2-e68b-4047-b1f6-edd838aaceb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:14.281765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59280fc2-e68b-4047-b1f6-edd838aaceb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:14.299087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59280fc2-e68b-4047-b1f6-edd838aaceb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:14.301190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59280fc2-e68b-4047-b1f6-edd838aaceb7 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:14.320786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1315/2624] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:43:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:43:14.558560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b97b3-f5ce-4eb2-8d97-096d5069acdc None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:14.561919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-066b97b3-f5ce-4eb2-8d97-096d5069acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:14.586050 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b97b3-f5ce-4eb2-8d97-096d5069acdc None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:14.592069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:14.593062 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:14.593408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:14.593806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] PATH_INFO: `/v3/projects/a713cd9e5960426a9378c54dd45e4997` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:14.603323 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a713cd9e5960426a9378c54dd45e4997)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:14.605414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:14.655803 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:43:14.657229 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a713cd9e5960426a9378c54dd45e4997', 'request_id': 'req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.666965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a713cd9e5960426a9378c54dd45e4997', 'request_id': 'req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.675703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a713cd9e5960426a9378c54dd45e4997', 'request_id': 'req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.689469 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] The token cache is being invalidate because project a713cd9e5960426a9378c54dd45e4997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:43:14.689739 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.690535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-3b83ed77-85b8-43a6-ad3d-0b9b6ad594d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:43:14.692151 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1310/2625] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:43:14 2022] DELETE /identity/v3/projects/a713cd9e5960426a9378c54dd45e4997 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:43:18.113068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89084f-06ca-41f7-b56f-7f4621c366a9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:18.113618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89084f-06ca-41f7-b56f-7f4621c366a9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:18.113618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89084f-06ca-41f7-b56f-7f4621c366a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:18.132612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fe89084f-06ca-41f7-b56f-7f4621c366a9 None None] MFA Rules not processed for user `868e2f0e822b430981369d414eee8e53`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:18.134098 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fe89084f-06ca-41f7-b56f-7f4621c366a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:18.198481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1316/2626] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:43:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:24.686542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=7,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:43:28.190985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:43:29.908758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a08d01-13ce-4a2e-a1da-50c5af3bc8de None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:29.909543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a08d01-13ce-4a2e-a1da-50c5af3bc8de None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:29.909543 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a08d01-13ce-4a2e-a1da-50c5af3bc8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:29.937526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-77a08d01-13ce-4a2e-a1da-50c5af3bc8de None None] MFA Rules not processed for user `d42b42214d0d46bbb9fe90aed7f5e382`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:29.939377 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-77a08d01-13ce-4a2e-a1da-50c5af3bc8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:30.002271 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1311/2627] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:43:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:36.902256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673fd8b-85c5-48fb-a395-0be870e8a41f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:36.902761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673fd8b-85c5-48fb-a395-0be870e8a41f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:36.902871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673fd8b-85c5-48fb-a395-0be870e8a41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:36.922722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9673fd8b-85c5-48fb-a395-0be870e8a41f None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:36.923938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9673fd8b-85c5-48fb-a395-0be870e8a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:36.981854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1317/2628] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:43:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:36.994268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a08d01-13ce-4a2e-a1da-50c5af3bc8de None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:36.995798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-77a08d01-13ce-4a2e-a1da-50c5af3bc8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:37.071757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a08d01-13ce-4a2e-a1da-50c5af3bc8de None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:37.077777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb8fd8-1b6f-444a-a606-c1057d937e64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:37.078433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb8fd8-1b6f-444a-a606-c1057d937e64 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:37.078717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb8fd8-1b6f-444a-a606-c1057d937e64 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:37.078972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb8fd8-1b6f-444a-a606-c1057d937e64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:37.088776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb8fd8-1b6f-444a-a606-c1057d937e64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:37.090388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb8fd8-1b6f-444a-a606-c1057d937e64 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:37.098908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1312/2629] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:43:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:43:46.974812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=25 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:43:47.095832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=57 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:43:47.929147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673fd8b-85c5-48fb-a395-0be870e8a41f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:47.930703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9673fd8b-85c5-48fb-a395-0be870e8a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:47.993499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673fd8b-85c5-48fb-a395-0be870e8a41f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:47.997115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bb1588-aaff-424c-aa3f-2639770d6a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:47.997780 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb1588-aaff-424c-aa3f-2639770d6a59 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:47.998064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb1588-aaff-424c-aa3f-2639770d6a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:47.998337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bb1588-aaff-424c-aa3f-2639770d6a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:48.000190 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-30bb1588-aaff-424c-aa3f-2639770d6a59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:48.070844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb1588-aaff-424c-aa3f-2639770d6a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:48.072347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bb1588-aaff-424c-aa3f-2639770d6a59 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:48.082300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1318/2630] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:43:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:43:51.958982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760add1-6991-411a-8e93-3cbcd229fd2e None None] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:51.959526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760add1-6991-411a-8e93-3cbcd229fd2e None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:51.959647 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760add1-6991-411a-8e93-3cbcd229fd2e None None] PATH_INFO: `/` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:51.961130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1313/2631] 10.222.0.42 () {54 vars in 921 bytes} [Sat Jul 2 03:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 02 03:43:51.965913 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f3dc5-09dd-4434-a5e2-5c6be72ec796 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:51.966350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f3dc5-09dd-4434-a5e2-5c6be72ec796 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:51.966632 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f3dc5-09dd-4434-a5e2-5c6be72ec796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:51.986265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b97f3dc5-09dd-4434-a5e2-5c6be72ec796 None None] MFA Rules not processed for user `8b7bc20034974816905978dd365993f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:51.987518 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b97f3dc5-09dd-4434-a5e2-5c6be72ec796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:52.060853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1319/2632] 10.222.0.42 () {58 vars in 1018 bytes} [Sat Jul 2 03:43:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:43:52.073047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760add1-6991-411a-8e93-3cbcd229fd2e None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:52.074830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0760add1-6991-411a-8e93-3cbcd229fd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:52.132903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760add1-6991-411a-8e93-3cbcd229fd2e None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:52.136871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535dc2ed-4d21-46e3-a4f8-f81473049f5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:52.137522 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535dc2ed-4d21-46e3-a4f8-f81473049f5a service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:52.137801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535dc2ed-4d21-46e3-a4f8-f81473049f5a service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:52.138075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535dc2ed-4d21-46e3-a4f8-f81473049f5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:52.149159 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535dc2ed-4d21-46e3-a4f8-f81473049f5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:52.150609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535dc2ed-4d21-46e3-a4f8-f81473049f5a service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:52.161412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1314/2633] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:43:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:43:54.018140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2637720-ea3d-490e-8249-3e258f35620c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:54.018621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2637720-ea3d-490e-8249-3e258f35620c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:54.018720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2637720-ea3d-490e-8249-3e258f35620c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:54.037305 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b2637720-ea3d-490e-8249-3e258f35620c None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:43:54.038533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b2637720-ea3d-490e-8249-3e258f35620c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:43:54.046703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1320/2634] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:43:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:43:54.058914 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535dc2ed-4d21-46e3-a4f8-f81473049f5a service glance] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:43:54.070906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535dc2ed-4d21-46e3-a4f8-f81473049f5a service glance] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:43:54.077116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eeabad-f4d9-434a-9838-c28f77a2b0ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:43:54.077757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeabad-f4d9-434a-9838-c28f77a2b0ba service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:43:54.078036 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeabad-f4d9-434a-9838-c28f77a2b0ba service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:43:54.078343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeabad-f4d9-434a-9838-c28f77a2b0ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:43:54.087259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeabad-f4d9-434a-9838-c28f77a2b0ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:43:54.088878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeabad-f4d9-434a-9838-c28f77a2b0ba service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:43:54.097135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1315/2635] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:44:04.031884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:44:04.093996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=32 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:44:12.920046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8ac99-7606-41ee-8dee-43c00fd013f1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:44:12.920564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8ac99-7606-41ee-8dee-43c00fd013f1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:44:12.920564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8ac99-7606-41ee-8dee-43c00fd013f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:44:12.941399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e6e8ac99-7606-41ee-8dee-43c00fd013f1 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:44:12.942580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e6e8ac99-7606-41ee-8dee-43c00fd013f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:44:12.951412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1321/2636] 10.222.0.42 () {58 vars in 967 bytes} [Sat Jul 2 03:44:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:44:12.973166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eeabad-f4d9-434a-9838-c28f77a2b0ba service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:44:12.984766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eeabad-f4d9-434a-9838-c28f77a2b0ba service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:44:12.991205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc9dc1-1c37-4bce-9d3c-af2202f0ddbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:44:12.991878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc9dc1-1c37-4bce-9d3c-af2202f0ddbf service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:44:12.992175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc9dc1-1c37-4bce-9d3c-af2202f0ddbf service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:44:12.992448 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc9dc1-1c37-4bce-9d3c-af2202f0ddbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:44:13.001207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc9dc1-1c37-4bce-9d3c-af2202f0ddbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:44:13.002809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc9dc1-1c37-4bce-9d3c-af2202f0ddbf service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:44:13.011353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1316/2637] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:44:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:44:22.936004 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:44:23.008044 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:02.859535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8ac99-7606-41ee-8dee-43c00fd013f1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:02.861175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e6e8ac99-7606-41ee-8dee-43c00fd013f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:02.918036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8ac99-7606-41ee-8dee-43c00fd013f1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:02.922530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:02.923177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:02.923523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:02.923797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:02.926367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:02.945433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:02.946875 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:02.958506 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1322/2638] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:10.933223 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc9dc1-1c37-4bce-9d3c-af2202f0ddbf service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:10.943329 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc9dc1-1c37-4bce-9d3c-af2202f0ddbf service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:10.949692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:10.950611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:10.950991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:10.951375 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:10.954586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:10.970780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:10.972305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:10.983785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1317/2639] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:12.953447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=26 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:20.392520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:20.394231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:20.407565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c131e-8d46-4115-8da9-3adb79d4a7ea service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:20.411769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:20.412425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:20.412705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:20.412984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:20.415070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:20.482840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:20.484346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:20.494533 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1323/2640] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:20.615790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:20.617445 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:20.629203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2bd8e-6edd-4d53-b9ee-2b08bd43af05 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:20.633881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f44ac6-2934-4bd0-bea2-4d4a5e1fe30b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:20.634505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f44ac6-2934-4bd0-bea2-4d4a5e1fe30b service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:20.634790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f44ac6-2934-4bd0-bea2-4d4a5e1fe30b service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:20.635066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f44ac6-2934-4bd0-bea2-4d4a5e1fe30b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:20.637290 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-99f44ac6-2934-4bd0-bea2-4d4a5e1fe30b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:20.723043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f44ac6-2934-4bd0-bea2-4d4a5e1fe30b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:20.724524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f44ac6-2934-4bd0-bea2-4d4a5e1fe30b service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:20.743964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1318/2641] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:24.363031 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:24.364935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:24.389198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a14b3-fbd3-413c-9491-09f8787f55c2 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:24.394143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:24.395025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:24.395319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:24.395605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] PATH_INFO: `/v3/users/868e2f0e822b430981369d414eee8e53` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:24.414301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=868e2f0e822b430981369d414eee8e53)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:24.416336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:24.481906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868e2f0e822b430981369d414eee8e53'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.482556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868e2f0e822b430981369d414eee8e53'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.490181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868e2f0e822b430981369d414eee8e53'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.502702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868e2f0e822b430981369d414eee8e53'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.503557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e031e1f1-2e33-46b9-884f-e2f0b9fde741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868e2f0e822b430981369d414eee8e53'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.519347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1324/2642] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:45:24 2022] DELETE /identity/v3/users/868e2f0e822b430981369d414eee8e53 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:45:24.702233 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f44ac6-2934-4bd0-bea2-4d4a5e1fe30b service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:24.723180 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f44ac6-2934-4bd0-bea2-4d4a5e1fe30b service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:24.794390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:24.795305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:24.795603 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:24.795877 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] PATH_INFO: `/v3/projects/b44eff44cb504e1f9bf037303db24d1b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:24.807954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44eff44cb504e1f9bf037303db24d1b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:24.810248 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:24.866537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44eff44cb504e1f9bf037303db24d1b', 'request_id': 'req-0110f32f-a1a7-4a87-abf3-48053a38940a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.869615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:24.881738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44eff44cb504e1f9bf037303db24d1b', 'request_id': 'req-0110f32f-a1a7-4a87-abf3-48053a38940a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.891486 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44eff44cb504e1f9bf037303db24d1b', 'request_id': 'req-0110f32f-a1a7-4a87-abf3-48053a38940a'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.910220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] The token cache is being invalidate because project b44eff44cb504e1f9bf037303db24d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:45:24.910835 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.915190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:24.918089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1319/2643] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:45:24 2022] DELETE /identity/v3/projects/b44eff44cb504e1f9bf037303db24d1b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:45:24.923914 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568476a3-54ba-4873-a74a-00af5f733642 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:24.924463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568476a3-54ba-4873-a74a-00af5f733642 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:24.924860 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568476a3-54ba-4873-a74a-00af5f733642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:24.963240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-568476a3-54ba-4873-a74a-00af5f733642 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:45:24.964476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-568476a3-54ba-4873-a74a-00af5f733642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:25.018776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.020654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:25.033558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:25.053389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1325/2644] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:45:25.062198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c9f37-3d3a-446c-b591-7c2c621ef5c0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.062491 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c9f37-3d3a-446c-b591-7c2c621ef5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.062772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c9f37-3d3a-446c-b591-7c2c621ef5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.083469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ea2c9f37-3d3a-446c-b591-7c2c621ef5c0 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:45:25.084610 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ea2c9f37-3d3a-446c-b591-7c2c621ef5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:25.093685 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1326/2645] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:45:25.098099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c9f37-3d3a-446c-b591-7c2c621ef5c0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.103110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110f32f-a1a7-4a87-abf3-48053a38940a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.107103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046f7b6-4307-4794-a474-789f93148d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.107881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046f7b6-4307-4794-a474-789f93148d68 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.108276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c9f37-3d3a-446c-b591-7c2c621ef5c0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.108414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046f7b6-4307-4794-a474-789f93148d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.108768 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046f7b6-4307-4794-a474-789f93148d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.110740 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a046f7b6-4307-4794-a474-789f93148d68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:25.113637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dd377-a1ee-47c1-a8ec-d9d3b8e0e433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.114357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dd377-a1ee-47c1-a8ec-d9d3b8e0e433 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.114707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dd377-a1ee-47c1-a8ec-d9d3b8e0e433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.115048 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dd377-a1ee-47c1-a8ec-d9d3b8e0e433 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.115893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dd377-a1ee-47c1-a8ec-d9d3b8e0e433 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:45:25.116266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dd377-a1ee-47c1-a8ec-d9d3b8e0e433 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.118141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dd377-a1ee-47c1-a8ec-d9d3b8e0e433 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.122919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046f7b6-4307-4794-a474-789f93148d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.124753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046f7b6-4307-4794-a474-789f93148d68 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.138490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1327/2646] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:45:25.139673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dd377-a1ee-47c1-a8ec-d9d3b8e0e433 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.141612 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1320/2647] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:25.150744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dd377-a1ee-47c1-a8ec-d9d3b8e0e433 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.156256 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5103c44-0fae-4d5a-8f57-4fff9b079fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.156925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5103c44-0fae-4d5a-8f57-4fff9b079fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.157215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5103c44-0fae-4d5a-8f57-4fff9b079fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.157484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5103c44-0fae-4d5a-8f57-4fff9b079fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.158174 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5103c44-0fae-4d5a-8f57-4fff9b079fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.160457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5103c44-0fae-4d5a-8f57-4fff9b079fb0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.192236 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1328/2648] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:45:25 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:45:25.196454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046f7b6-4307-4794-a474-789f93148d68 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.208899 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046f7b6-4307-4794-a474-789f93148d68 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.287166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db23f7b-07f5-47da-bb21-9be5d2c4645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.287867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db23f7b-07f5-47da-bb21-9be5d2c4645d admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.291266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db23f7b-07f5-47da-bb21-9be5d2c4645d admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.291929 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db23f7b-07f5-47da-bb21-9be5d2c4645d admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.292673 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db23f7b-07f5-47da-bb21-9be5d2c4645d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.294819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db23f7b-07f5-47da-bb21-9be5d2c4645d admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.329903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1321/2649] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:45:25 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:45:25.334727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5103c44-0fae-4d5a-8f57-4fff9b079fb0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.345734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5103c44-0fae-4d5a-8f57-4fff9b079fb0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.350824 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71b89d-d53e-484b-95ae-7403ab0489bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.351544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71b89d-d53e-484b-95ae-7403ab0489bd admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.351831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71b89d-d53e-484b-95ae-7403ab0489bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.352114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71b89d-d53e-484b-95ae-7403ab0489bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.353181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e71b89d-d53e-484b-95ae-7403ab0489bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.354466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e71b89d-d53e-484b-95ae-7403ab0489bd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.370175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1329/2650] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:45:25.374809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db23f7b-07f5-47da-bb21-9be5d2c4645d admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.386597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db23f7b-07f5-47da-bb21-9be5d2c4645d admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.392611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef17937-8329-4ce7-828c-0c861119de78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.393387 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef17937-8329-4ce7-828c-0c861119de78 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.394076 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef17937-8329-4ce7-828c-0c861119de78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.394433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef17937-8329-4ce7-828c-0c861119de78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.395790 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef17937-8329-4ce7-828c-0c861119de78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.397548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef17937-8329-4ce7-828c-0c861119de78 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.408585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1322/2651] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:45:25.412975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71b89d-d53e-484b-95ae-7403ab0489bd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.423904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71b89d-d53e-484b-95ae-7403ab0489bd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.430037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d78d-808b-45ea-af09-9d5d8a72cafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.431017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d78d-808b-45ea-af09-9d5d8a72cafb admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.431411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d78d-808b-45ea-af09-9d5d8a72cafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.431754 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d78d-808b-45ea-af09-9d5d8a72cafb admin admin] PATH_INFO: `/v3/projects/c81af44a6ff34d3baaccb30419cebb40/users/2d804275cda84bfeab7c5b057cb703cf/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.449481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70d78d-808b-45ea-af09-9d5d8a72cafb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81af44a6ff34d3baaccb30419cebb40, user_id=2d804275cda84bfeab7c5b057cb703cf, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.452376 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70d78d-808b-45ea-af09-9d5d8a72cafb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.467590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1330/2652] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:45:25 2022] PUT /identity/v3/projects/c81af44a6ff34d3baaccb30419cebb40/users/2d804275cda84bfeab7c5b057cb703cf/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:45:25.480378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94665f-32d2-4303-ae98-01c4962321e1 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.480842 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94665f-32d2-4303-ae98-01c4962321e1 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.481301 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94665f-32d2-4303-ae98-01c4962321e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.511415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1b94665f-32d2-4303-ae98-01c4962321e1 None None] MFA Rules not processed for user `2d804275cda84bfeab7c5b057cb703cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:45:25.512621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1b94665f-32d2-4303-ae98-01c4962321e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:25.562736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:25.583456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1323/2653] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:45:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:45:25.587945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d78d-808b-45ea-af09-9d5d8a72cafb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.650804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d78d-808b-45ea-af09-9d5d8a72cafb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.655968 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55439bed-de59-4218-808c-95596b6b7562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.656630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55439bed-de59-4218-808c-95596b6b7562 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.656891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55439bed-de59-4218-808c-95596b6b7562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.657168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55439bed-de59-4218-808c-95596b6b7562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.657887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55439bed-de59-4218-808c-95596b6b7562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.660459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55439bed-de59-4218-808c-95596b6b7562 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.692100 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1331/2654] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:45:25 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:45:25.693297 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94665f-32d2-4303-ae98-01c4962321e1 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.704614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94665f-32d2-4303-ae98-01c4962321e1 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.772614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f02bf-9b45-4616-9a97-8f998c2266bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.773517 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f02bf-9b45-4616-9a97-8f998c2266bf admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.773907 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f02bf-9b45-4616-9a97-8f998c2266bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.774274 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f02bf-9b45-4616-9a97-8f998c2266bf admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.775082 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f02bf-9b45-4616-9a97-8f998c2266bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.777155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f02bf-9b45-4616-9a97-8f998c2266bf admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.802149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1324/2655] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:45:25 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:45:25.803508 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55439bed-de59-4218-808c-95596b6b7562 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.817774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55439bed-de59-4218-808c-95596b6b7562 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.822628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f64b81-4e88-474c-b4ea-95f8ddd4dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.823761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f64b81-4e88-474c-b4ea-95f8ddd4dcb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.824046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f64b81-4e88-474c-b4ea-95f8ddd4dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.824321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f64b81-4e88-474c-b4ea-95f8ddd4dcb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.825051 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f64b81-4e88-474c-b4ea-95f8ddd4dcb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.826670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f64b81-4e88-474c-b4ea-95f8ddd4dcb4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.842287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1332/2656] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:45:25.843552 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f02bf-9b45-4616-9a97-8f998c2266bf admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.853571 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f02bf-9b45-4616-9a97-8f998c2266bf admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.858738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dab5d-224c-4cb9-bea5-00bb64ecd184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.859680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dab5d-224c-4cb9-bea5-00bb64ecd184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.859963 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dab5d-224c-4cb9-bea5-00bb64ecd184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.860237 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612dab5d-224c-4cb9-bea5-00bb64ecd184 admin admin] PATH_INFO: `/v3/projects/58b10e027442413ab48eb2f8823c6d23/users/7fdba9f8673847deb44362a75e97f036/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.876162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dab5d-224c-4cb9-bea5-00bb64ecd184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b10e027442413ab48eb2f8823c6d23, user_id=7fdba9f8673847deb44362a75e97f036, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.879083 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612dab5d-224c-4cb9-bea5-00bb64ecd184 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.895281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1325/2657] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:45:25 2022] PUT /identity/v3/projects/58b10e027442413ab48eb2f8823c6d23/users/7fdba9f8673847deb44362a75e97f036/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:45:25.900526 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f64b81-4e88-474c-b4ea-95f8ddd4dcb4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:25.911617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f64b81-4e88-474c-b4ea-95f8ddd4dcb4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:25.966462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:25.977833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398f91d-7779-4454-b065-9c58415ff74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:25.978522 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398f91d-7779-4454-b065-9c58415ff74e admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:25.978806 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398f91d-7779-4454-b065-9c58415ff74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:25.979080 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398f91d-7779-4454-b065-9c58415ff74e admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:25.979844 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398f91d-7779-4454-b065-9c58415ff74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:25.982614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398f91d-7779-4454-b065-9c58415ff74e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:25.995324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1333/2658] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:45:25.998656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dab5d-224c-4cb9-bea5-00bb64ecd184 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:26.009588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612dab5d-224c-4cb9-bea5-00bb64ecd184 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:26.014335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877d3e5-d5c3-4c8e-baab-1132074d7dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:26.015681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877d3e5-d5c3-4c8e-baab-1132074d7dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:26.016167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877d3e5-d5c3-4c8e-baab-1132074d7dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:26.016602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877d3e5-d5c3-4c8e-baab-1132074d7dcc admin admin] PATH_INFO: `/v3/projects/58b10e027442413ab48eb2f8823c6d23/users/7fdba9f8673847deb44362a75e97f036/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:26.019659 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877d3e5-d5c3-4c8e-baab-1132074d7dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b10e027442413ab48eb2f8823c6d23, user_id=7fdba9f8673847deb44362a75e97f036, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:26.023775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877d3e5-d5c3-4c8e-baab-1132074d7dcc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:26.034955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1326/2659] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:45:25 2022] PUT /identity/v3/projects/58b10e027442413ab48eb2f8823c6d23/users/7fdba9f8673847deb44362a75e97f036/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:45:26.045639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12409b-ff38-4c43-a9e4-b6f08e93bff5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:26.045931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12409b-ff38-4c43-a9e4-b6f08e93bff5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:26.046215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12409b-ff38-4c43-a9e4-b6f08e93bff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:26.075227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fe12409b-ff38-4c43-a9e4-b6f08e93bff5 None None] MFA Rules not processed for user `7fdba9f8673847deb44362a75e97f036`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:45:26.076554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fe12409b-ff38-4c43-a9e4-b6f08e93bff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:26.158060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1334/2660] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:45:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:45:26.169621 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877d3e5-d5c3-4c8e-baab-1132074d7dcc admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:26.171401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2877d3e5-d5c3-4c8e-baab-1132074d7dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:26.230128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877d3e5-d5c3-4c8e-baab-1132074d7dcc admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:26.233912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a286ad2-48a0-45ce-82fd-e0e988bf0810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:26.234566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a286ad2-48a0-45ce-82fd-e0e988bf0810 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:26.234857 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a286ad2-48a0-45ce-82fd-e0e988bf0810 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:26.235121 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a286ad2-48a0-45ce-82fd-e0e988bf0810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:26.293645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:26.296342 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a286ad2-48a0-45ce-82fd-e0e988bf0810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:26.297979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a286ad2-48a0-45ce-82fd-e0e988bf0810 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:26.309738 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1327/2661] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:28.874203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12409b-ff38-4c43-a9e4-b6f08e93bff5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:28.876692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fe12409b-ff38-4c43-a9e4-b6f08e93bff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:28.941883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12409b-ff38-4c43-a9e4-b6f08e93bff5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:28.945947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bde33-9a62-463b-9b57-afae6886db33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:28.946598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bde33-9a62-463b-9b57-afae6886db33 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:28.946877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bde33-9a62-463b-9b57-afae6886db33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:28.947160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bde33-9a62-463b-9b57-afae6886db33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:28.949148 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e76bde33-9a62-463b-9b57-afae6886db33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:29.011951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bde33-9a62-463b-9b57-afae6886db33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:29.013399 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bde33-9a62-463b-9b57-afae6886db33 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:29.024019 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1335/2662] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:32.376409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a286ad2-48a0-45ce-82fd-e0e988bf0810 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:32.377859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9a286ad2-48a0-45ce-82fd-e0e988bf0810 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:32.392305 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a286ad2-48a0-45ce-82fd-e0e988bf0810 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:32.397205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:32.398116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:32.398396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:32.398674 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] PATH_INFO: `/v3/users/112d731047a64efc8e9143c4d471a7d6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:32.399936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=112d731047a64efc8e9143c4d471a7d6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:32.401961 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:32.451384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '112d731047a64efc8e9143c4d471a7d6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.452118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '112d731047a64efc8e9143c4d471a7d6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.461464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '112d731047a64efc8e9143c4d471a7d6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.462139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '112d731047a64efc8e9143c4d471a7d6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.474864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-747eacb4-4a8b-4476-80e1-0434348d0b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '112d731047a64efc8e9143c4d471a7d6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.486976 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1328/2663] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:45:32 2022] DELETE /identity/v3/users/112d731047a64efc8e9143c4d471a7d6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:45:32.674666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bde33-9a62-463b-9b57-afae6886db33 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:32.685297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bde33-9a62-463b-9b57-afae6886db33 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:32.708981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:32.747095 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:32.748009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:32.748289 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:32.748559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] PATH_INFO: `/v3/projects/7f39e852f8d14dec8e98d8d77fe90766` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:32.749790 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f39e852f8d14dec8e98d8d77fe90766)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:32.751749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:32.797242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f39e852f8d14dec8e98d8d77fe90766', 'request_id': 'req-6f41a808-dee6-40b1-a1ee-e93e23247d4d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.803703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f39e852f8d14dec8e98d8d77fe90766', 'request_id': 'req-6f41a808-dee6-40b1-a1ee-e93e23247d4d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.810105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f39e852f8d14dec8e98d8d77fe90766', 'request_id': 'req-6f41a808-dee6-40b1-a1ee-e93e23247d4d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.825514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] The token cache is being invalidate because project 7f39e852f8d14dec8e98d8d77fe90766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:45:32.825797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.826472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6f41a808-dee6-40b1-a1ee-e93e23247d4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:45:32.828125 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1336/2664] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:45:32 2022] DELETE /identity/v3/projects/7f39e852f8d14dec8e98d8d77fe90766 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:45:32.834099 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a46f68-378d-4a1f-bd9f-f1f84156970b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:32.834389 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a46f68-378d-4a1f-bd9f-f1f84156970b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:32.834665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a46f68-378d-4a1f-bd9f-f1f84156970b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:32.852588 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-72a46f68-378d-4a1f-bd9f-f1f84156970b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:45:32.853792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-72a46f68-378d-4a1f-bd9f-f1f84156970b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:32.920217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1329/2665] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:45:32.928614 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53787d-15c5-4a0c-99e5-0f37968f07dd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:32.928906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53787d-15c5-4a0c-99e5-0f37968f07dd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:32.929186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53787d-15c5-4a0c-99e5-0f37968f07dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:32.950463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bc53787d-15c5-4a0c-99e5-0f37968f07dd None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:45:32.951741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bc53787d-15c5-4a0c-99e5-0f37968f07dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:32.961722 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1337/2666] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:45:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:45:32.966615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a46f68-378d-4a1f-bd9f-f1f84156970b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:32.976696 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a46f68-378d-4a1f-bd9f-f1f84156970b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:32.981431 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:32.982127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:32.982411 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:32.982705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:32.983436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:45:32.983881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:32.985502 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:32.998879 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1330/2667] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:45:33.003197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53787d-15c5-4a0c-99e5-0f37968f07dd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:33.015428 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53787d-15c5-4a0c-99e5-0f37968f07dd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:33.021456 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bb20a-5838-493a-b098-d6e7b98f97f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:33.022135 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bb20a-5838-493a-b098-d6e7b98f97f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:33.022421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bb20a-5838-493a-b098-d6e7b98f97f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:33.022702 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bb20a-5838-493a-b098-d6e7b98f97f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:33.023433 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bb20a-5838-493a-b098-d6e7b98f97f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:33.025463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bb20a-5838-493a-b098-d6e7b98f97f6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:33.026459 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:33.028205 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:33.055349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1338/2668] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:45:33 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:45:33.057809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bb20a-5838-493a-b098-d6e7b98f97f6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:33.084927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5f5dd-e18b-4c49-807e-03ebb26ce268 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:33.092758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:33.121956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bb20a-5838-493a-b098-d6e7b98f97f6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:33.129561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe25ab-e099-4a7d-885b-bc0f51e8e5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:33.129561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe25ab-e099-4a7d-885b-bc0f51e8e5f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:33.129561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe25ab-e099-4a7d-885b-bc0f51e8e5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:33.129561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe25ab-e099-4a7d-885b-bc0f51e8e5f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:33.129967 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe25ab-e099-4a7d-885b-bc0f51e8e5f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:33.131557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe25ab-e099-4a7d-885b-bc0f51e8e5f4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:33.139087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bd093-66ee-4842-82a6-ea84703836a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:33.140197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd093-66ee-4842-82a6-ea84703836a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:33.140734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd093-66ee-4842-82a6-ea84703836a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:33.141046 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd093-66ee-4842-82a6-ea84703836a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:33.143098 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ec8bd093-66ee-4842-82a6-ea84703836a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:33.151837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1339/2669] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:45:33 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:45:33.155866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe25ab-e099-4a7d-885b-bc0f51e8e5f4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:33.165197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe25ab-e099-4a7d-885b-bc0f51e8e5f4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:33.170555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517e521-d01b-4365-8724-529f0290dc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:33.171142 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517e521-d01b-4365-8724-529f0290dc5c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:33.171449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517e521-d01b-4365-8724-529f0290dc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:33.171724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517e521-d01b-4365-8724-529f0290dc5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:33.172440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517e521-d01b-4365-8724-529f0290dc5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:33.174058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517e521-d01b-4365-8724-529f0290dc5c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:33.184315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1340/2670] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:45:33.188348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517e521-d01b-4365-8724-529f0290dc5c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:33.194557 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:33.198695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517e521-d01b-4365-8724-529f0290dc5c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:33.204803 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d4451-6075-4341-a966-dfa458df7eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:33.205565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d4451-6075-4341-a966-dfa458df7eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:33.206017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d4451-6075-4341-a966-dfa458df7eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:33.206270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d4451-6075-4341-a966-dfa458df7eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:33.206993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d4451-6075-4341-a966-dfa458df7eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:33.209358 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d4451-6075-4341-a966-dfa458df7eb2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:33.214787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bd093-66ee-4842-82a6-ea84703836a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:33.216284 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bd093-66ee-4842-82a6-ea84703836a7 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:33.220150 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1341/2671] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:45:33.224276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d4451-6075-4341-a966-dfa458df7eb2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:33.229678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1331/2672] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:33.234416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d4451-6075-4341-a966-dfa458df7eb2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:33.239828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091de87-f26e-43b5-8d56-a3eb6ee0756b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:33.240716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091de87-f26e-43b5-8d56-a3eb6ee0756b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:33.240997 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091de87-f26e-43b5-8d56-a3eb6ee0756b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:33.241269 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091de87-f26e-43b5-8d56-a3eb6ee0756b admin admin] PATH_INFO: `/v3/projects/0665267ba574490cbcf958be4d793dd3/users/e80e09005b7e4cdc8f42be866dbd91bb/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:33.256834 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091de87-f26e-43b5-8d56-a3eb6ee0756b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0665267ba574490cbcf958be4d793dd3, user_id=e80e09005b7e4cdc8f42be866dbd91bb, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:33.259516 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091de87-f26e-43b5-8d56-a3eb6ee0756b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:33.273978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1342/2673] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:45:33 2022] PUT /identity/v3/projects/0665267ba574490cbcf958be4d793dd3/users/e80e09005b7e4cdc8f42be866dbd91bb/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:45:33.279968 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70044d9e-a188-4ee4-b071-ddb247355068 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:33.280473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70044d9e-a188-4ee4-b071-ddb247355068 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:33.280955 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70044d9e-a188-4ee4-b071-ddb247355068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:33.298455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-70044d9e-a188-4ee4-b071-ddb247355068 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:45:33.299723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-70044d9e-a188-4ee4-b071-ddb247355068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:33.368470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1332/2674] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:45:33.378488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091de87-f26e-43b5-8d56-a3eb6ee0756b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:33.380517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5091de87-f26e-43b5-8d56-a3eb6ee0756b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:33.442705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091de87-f26e-43b5-8d56-a3eb6ee0756b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:33.446467 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6f32e-4931-4c93-aed3-a55c8d59ed6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:33.447123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6f32e-4931-4c93-aed3-a55c8d59ed6f service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:33.447440 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6f32e-4931-4c93-aed3-a55c8d59ed6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:33.447719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de6f32e-4931-4c93-aed3-a55c8d59ed6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:33.457223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6f32e-4931-4c93-aed3-a55c8d59ed6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:33.459398 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de6f32e-4931-4c93-aed3-a55c8d59ed6f service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:33.469607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1343/2675] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:37.552396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0be0c0-fccb-49db-9fc7-b7c13ad53450 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:37.552936 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0be0c0-fccb-49db-9fc7-b7c13ad53450 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:37.553055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0be0c0-fccb-49db-9fc7-b7c13ad53450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:37.579893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-2a0be0c0-fccb-49db-9fc7-b7c13ad53450 None None] MFA Rules not processed for user `e80e09005b7e4cdc8f42be866dbd91bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:45:37.581213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2a0be0c0-fccb-49db-9fc7-b7c13ad53450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:37.634757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:37.671223 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1333/2676] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:45:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:45:37.685885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6f32e-4931-4c93-aed3-a55c8d59ed6f service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:37.687645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9de6f32e-4931-4c93-aed3-a55c8d59ed6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:37.768347 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de6f32e-4931-4c93-aed3-a55c8d59ed6f service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:37.773970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2b140-218d-4a66-8a7c-b0172928f302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:37.774647 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b140-218d-4a66-8a7c-b0172928f302 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:37.774931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b140-218d-4a66-8a7c-b0172928f302 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:37.775205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b140-218d-4a66-8a7c-b0172928f302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:37.784648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2b140-218d-4a66-8a7c-b0172928f302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:37.786577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2b140-218d-4a66-8a7c-b0172928f302 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:37.797797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1344/2677] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:39.196939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bec90-351e-4c3f-b8d2-1bf5cc32a9a6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:39.197418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bec90-351e-4c3f-b8d2-1bf5cc32a9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:39.197551 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bec90-351e-4c3f-b8d2-1bf5cc32a9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:39.216208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-474bec90-351e-4c3f-b8d2-1bf5cc32a9a6 None None] MFA Rules not processed for user `e80e09005b7e4cdc8f42be866dbd91bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:45:39.217434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-474bec90-351e-4c3f-b8d2-1bf5cc32a9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:39.224767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1334/2678] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:45:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:45:40.887450 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2b140-218d-4a66-8a7c-b0172928f302 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:40.889374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3ba2b140-218d-4a66-8a7c-b0172928f302 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:40.993856 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:41.009288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2b140-218d-4a66-8a7c-b0172928f302 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:41.015120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def83d4-c4cc-43f5-9267-449625e73078 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:41.015898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def83d4-c4cc-43f5-9267-449625e73078 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:41.016257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def83d4-c4cc-43f5-9267-449625e73078 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:41.016604 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def83d4-c4cc-43f5-9267-449625e73078 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:41.018837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9def83d4-c4cc-43f5-9267-449625e73078 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:41.032799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def83d4-c4cc-43f5-9267-449625e73078 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:41.034468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def83d4-c4cc-43f5-9267-449625e73078 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:41.046594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1345/2679] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:46.231748 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bec90-351e-4c3f-b8d2-1bf5cc32a9a6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:46.233360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-474bec90-351e-4c3f-b8d2-1bf5cc32a9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:46.260185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bec90-351e-4c3f-b8d2-1bf5cc32a9a6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:46.266055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:46.266903 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:46.267285 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:46.267704 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:46.269940 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:46.283856 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:46.285589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:46.298441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1335/2680] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:45:51.041157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:56.293362 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=13 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:45:59.448957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def83d4-c4cc-43f5-9267-449625e73078 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:45:59.463629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def83d4-c4cc-43f5-9267-449625e73078 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:45:59.469931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599bcfd0-95fe-424f-b1e9-7287f32b1adf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:45:59.470598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bcfd0-95fe-424f-b1e9-7287f32b1adf service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:45:59.470883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bcfd0-95fe-424f-b1e9-7287f32b1adf service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:45:59.471159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599bcfd0-95fe-424f-b1e9-7287f32b1adf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:45:59.473625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-599bcfd0-95fe-424f-b1e9-7287f32b1adf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:45:59.488036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bcfd0-95fe-424f-b1e9-7287f32b1adf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:45:59.489664 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599bcfd0-95fe-424f-b1e9-7287f32b1adf service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:45:59.501659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1346/2681] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:45:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:02.378875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:02.380568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:02.395722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a572c2-a109-4c66-8bf0-dcca8e1a0a1e service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:02.400655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:02.401537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:02.401796 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:02.402093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] PATH_INFO: `/v3/users/2d804275cda84bfeab7c5b057cb703cf` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:02.403363 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d804275cda84bfeab7c5b057cb703cf)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:02.405311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:02.456423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d804275cda84bfeab7c5b057cb703cf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:02.457137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d804275cda84bfeab7c5b057cb703cf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:02.466011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d804275cda84bfeab7c5b057cb703cf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:02.466773 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d804275cda84bfeab7c5b057cb703cf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:02.484777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d804275cda84bfeab7c5b057cb703cf'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:02.497386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1336/2682] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:02 2022] DELETE /identity/v3/users/2d804275cda84bfeab7c5b057cb703cf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:02.502492 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22801-3652-4f07-bdeb-1dd87e87e208 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:02.503111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22801-3652-4f07-bdeb-1dd87e87e208 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:02.503111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22801-3652-4f07-bdeb-1dd87e87e208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:02.524916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-79d22801-3652-4f07-bdeb-1dd87e87e208 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:02.526110 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-79d22801-3652-4f07-bdeb-1dd87e87e208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:02.596877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1347/2683] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:02.607834 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:02.609627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:02.670825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa494d-1f0a-43ba-98f9-a68dc14d83ab admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:02.675196 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c6aa7-7eab-41cc-a034-8b26a1541149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:02.675883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c6aa7-7eab-41cc-a034-8b26a1541149 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:02.676162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c6aa7-7eab-41cc-a034-8b26a1541149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:02.676423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c6aa7-7eab-41cc-a034-8b26a1541149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:02.686410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c6aa7-7eab-41cc-a034-8b26a1541149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:02.687985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c6aa7-7eab-41cc-a034-8b26a1541149 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:02.697775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1337/2684] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:02.968394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22801-3652-4f07-bdeb-1dd87e87e208 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:02.970060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-79d22801-3652-4f07-bdeb-1dd87e87e208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:02.984356 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22801-3652-4f07-bdeb-1dd87e87e208 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:02.990021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:02.991324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:02.991713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:02.992085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] PATH_INFO: `/v3/projects/c81af44a6ff34d3baaccb30419cebb40` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:02.993581 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c81af44a6ff34d3baaccb30419cebb40)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:02.996271 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:03.048187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81af44a6ff34d3baaccb30419cebb40', 'request_id': 'req-78594a0c-dd77-4023-be0f-cc703ae3f6f3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.053111 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81af44a6ff34d3baaccb30419cebb40', 'request_id': 'req-78594a0c-dd77-4023-be0f-cc703ae3f6f3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.059272 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81af44a6ff34d3baaccb30419cebb40', 'request_id': 'req-78594a0c-dd77-4023-be0f-cc703ae3f6f3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.074965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] The token cache is being invalidate because project c81af44a6ff34d3baaccb30419cebb40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:03.075407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.076102 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.077677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1348/2685] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:02 2022] DELETE /identity/v3/projects/c81af44a6ff34d3baaccb30419cebb40 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:03.082009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c6aa7-7eab-41cc-a034-8b26a1541149 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:03.091518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c6aa7-7eab-41cc-a034-8b26a1541149 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:03.150003 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:03.150883 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:03.151162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:03.151477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] PATH_INFO: `/v3/users/7fdba9f8673847deb44362a75e97f036` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:03.152791 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fdba9f8673847deb44362a75e97f036)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:03.154699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:03.171777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:03.205665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fdba9f8673847deb44362a75e97f036'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.206494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fdba9f8673847deb44362a75e97f036'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.215005 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fdba9f8673847deb44362a75e97f036'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.215757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fdba9f8673847deb44362a75e97f036'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.229064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-702df40d-5436-4f92-ae5a-71098d659e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fdba9f8673847deb44362a75e97f036'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.241470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1338/2686] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:03 2022] DELETE /identity/v3/users/7fdba9f8673847deb44362a75e97f036 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:03.523730 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:03.525475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:03.541213 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78594a0c-dd77-4023-be0f-cc703ae3f6f3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:03.603322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:03.604417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:03.604707 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:03.604985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] PATH_INFO: `/v3/projects/58b10e027442413ab48eb2f8823c6d23` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:03.606180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] RBAC: Authorizing `identity:delete_project(project_id=58b10e027442413ab48eb2f8823c6d23)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:03.608179 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:03.620280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:03.659573 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58b10e027442413ab48eb2f8823c6d23', 'request_id': 'req-895f2b91-d544-4483-bd1b-47dd7c4e171a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.664905 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58b10e027442413ab48eb2f8823c6d23', 'request_id': 'req-895f2b91-d544-4483-bd1b-47dd7c4e171a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.671544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58b10e027442413ab48eb2f8823c6d23', 'request_id': 'req-895f2b91-d544-4483-bd1b-47dd7c4e171a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.686231 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] The token cache is being invalidate because project 58b10e027442413ab48eb2f8823c6d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:03.686535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.687151 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-895f2b91-d544-4483-bd1b-47dd7c4e171a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:03.689106 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1349/2687] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:03 2022] DELETE /identity/v3/projects/58b10e027442413ab48eb2f8823c6d23 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:03.694017 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67121-265c-4703-bddf-fb89beab1a44 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:03.694546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67121-265c-4703-bddf-fb89beab1a44 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:03.694648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e67121-265c-4703-bddf-fb89beab1a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:03.715209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-51e67121-265c-4703-bddf-fb89beab1a44 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:03.716504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-51e67121-265c-4703-bddf-fb89beab1a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:03.777939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1339/2688] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:46:03.786851 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4aba6-2047-497c-a9f3-b8effd9128da None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:03.787171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4aba6-2047-497c-a9f3-b8effd9128da None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:03.787482 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4aba6-2047-497c-a9f3-b8effd9128da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:03.807841 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-09d4aba6-2047-497c-a9f3-b8effd9128da None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:03.809066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-09d4aba6-2047-497c-a9f3-b8effd9128da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:03.817292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1350/2689] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:03.821687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e67121-265c-4703-bddf-fb89beab1a44 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:03.831665 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e67121-265c-4703-bddf-fb89beab1a44 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:03.836476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc00366-806a-4177-a8ee-46d19f940131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:03.837164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc00366-806a-4177-a8ee-46d19f940131 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:03.837443 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc00366-806a-4177-a8ee-46d19f940131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:03.837701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc00366-806a-4177-a8ee-46d19f940131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:03.838423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc00366-806a-4177-a8ee-46d19f940131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:46:03.838743 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc00366-806a-4177-a8ee-46d19f940131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:03.840395 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc00366-806a-4177-a8ee-46d19f940131 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:03.852253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1340/2690] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:46:03.856700 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4aba6-2047-497c-a9f3-b8effd9128da None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:03.866791 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4aba6-2047-497c-a9f3-b8effd9128da None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:03.871630 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f128422-ac83-4749-9c5f-3eff9bb43475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:03.872285 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128422-ac83-4749-9c5f-3eff9bb43475 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:03.872567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128422-ac83-4749-9c5f-3eff9bb43475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:03.872845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f128422-ac83-4749-9c5f-3eff9bb43475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:03.873535 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f128422-ac83-4749-9c5f-3eff9bb43475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:03.875502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f128422-ac83-4749-9c5f-3eff9bb43475 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:03.902628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1351/2691] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:03 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:03.906880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc00366-806a-4177-a8ee-46d19f940131 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:03.917299 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc00366-806a-4177-a8ee-46d19f940131 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:03.973385 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3542ada-0975-4f0d-9314-53c5a3f937a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:03.974090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3542ada-0975-4f0d-9314-53c5a3f937a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:03.974393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3542ada-0975-4f0d-9314-53c5a3f937a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:03.974656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3542ada-0975-4f0d-9314-53c5a3f937a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:03.975506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3542ada-0975-4f0d-9314-53c5a3f937a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:03.977585 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3542ada-0975-4f0d-9314-53c5a3f937a1 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.002910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1341/2692] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:03 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:04.004437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f128422-ac83-4749-9c5f-3eff9bb43475 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.015185 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f128422-ac83-4749-9c5f-3eff9bb43475 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.021064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e5d78-c9fe-4622-9f38-327f664727b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.021845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e5d78-c9fe-4622-9f38-327f664727b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.022245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e5d78-c9fe-4622-9f38-327f664727b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.022662 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e5d78-c9fe-4622-9f38-327f664727b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.023341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e5d78-c9fe-4622-9f38-327f664727b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.024988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e5d78-c9fe-4622-9f38-327f664727b5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.036107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1352/2693] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:04.040293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3542ada-0975-4f0d-9314-53c5a3f937a1 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.051026 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3542ada-0975-4f0d-9314-53c5a3f937a1 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.056382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951828fb-c9d7-45ec-9e78-a59510eb3d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.057052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951828fb-c9d7-45ec-9e78-a59510eb3d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.057334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951828fb-c9d7-45ec-9e78-a59510eb3d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.057610 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951828fb-c9d7-45ec-9e78-a59510eb3d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.058333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951828fb-c9d7-45ec-9e78-a59510eb3d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.059985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951828fb-c9d7-45ec-9e78-a59510eb3d0c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.065701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:04.071983 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1342/2694] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:04.076297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e5d78-c9fe-4622-9f38-327f664727b5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.086580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e5d78-c9fe-4622-9f38-327f664727b5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.091223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2d521-a6fa-4ecb-a459-13357be73e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.092147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2d521-a6fa-4ecb-a459-13357be73e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.092425 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2d521-a6fa-4ecb-a459-13357be73e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.092697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2d521-a6fa-4ecb-a459-13357be73e41 admin admin] PATH_INFO: `/v3/projects/7f958221d01d48ff9955895e76825fc0/users/5753a340076847828322c3d9e84e16fe/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.111195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2d521-a6fa-4ecb-a459-13357be73e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f958221d01d48ff9955895e76825fc0, user_id=5753a340076847828322c3d9e84e16fe, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.114042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2d521-a6fa-4ecb-a459-13357be73e41 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.134127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1353/2695] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:04 2022] PUT /identity/v3/projects/7f958221d01d48ff9955895e76825fc0/users/5753a340076847828322c3d9e84e16fe/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:04.144425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5fbc-8a5e-4898-b524-37ed0b089632 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.144729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5fbc-8a5e-4898-b524-37ed0b089632 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.145350 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5fbc-8a5e-4898-b524-37ed0b089632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.174029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-fa7f5fbc-8a5e-4898-b524-37ed0b089632 None None] MFA Rules not processed for user `5753a340076847828322c3d9e84e16fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:04.175408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fa7f5fbc-8a5e-4898-b524-37ed0b089632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:04.245613 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1343/2696] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:04.250077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2d521-a6fa-4ecb-a459-13357be73e41 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.322919 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2d521-a6fa-4ecb-a459-13357be73e41 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.327807 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3146d-8bc7-4c9a-aa3d-f176c0f0b3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.328585 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3146d-8bc7-4c9a-aa3d-f176c0f0b3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.329006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3146d-8bc7-4c9a-aa3d-f176c0f0b3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.329870 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3146d-8bc7-4c9a-aa3d-f176c0f0b3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.330199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3146d-8bc7-4c9a-aa3d-f176c0f0b3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.332258 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3146d-8bc7-4c9a-aa3d-f176c0f0b3d4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.367790 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1354/2697] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:04 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:04.369128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f5fbc-8a5e-4898-b524-37ed0b089632 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.385092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f5fbc-8a5e-4898-b524-37ed0b089632 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.453863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816a985-f6a0-440f-9b21-b48a7be5164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.454429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a985-f6a0-440f-9b21-b48a7be5164a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.454699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a985-f6a0-440f-9b21-b48a7be5164a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.454999 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a985-f6a0-440f-9b21-b48a7be5164a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.455865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816a985-f6a0-440f-9b21-b48a7be5164a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.457832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816a985-f6a0-440f-9b21-b48a7be5164a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.481089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1344/2698] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:04 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:04.485199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3146d-8bc7-4c9a-aa3d-f176c0f0b3d4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.497924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3146d-8bc7-4c9a-aa3d-f176c0f0b3d4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.504421 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69f659-120c-45ca-8c8e-370246d09de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.505618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69f659-120c-45ca-8c8e-370246d09de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.505895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69f659-120c-45ca-8c8e-370246d09de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.506168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69f659-120c-45ca-8c8e-370246d09de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.506897 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69f659-120c-45ca-8c8e-370246d09de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.508631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69f659-120c-45ca-8c8e-370246d09de5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.524230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1355/2699] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:46:04.528699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816a985-f6a0-440f-9b21-b48a7be5164a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.539516 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816a985-f6a0-440f-9b21-b48a7be5164a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.544750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e3302-6ef4-40e5-b206-08e553143f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.545651 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e3302-6ef4-40e5-b206-08e553143f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.545935 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e3302-6ef4-40e5-b206-08e553143f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.546242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e3302-6ef4-40e5-b206-08e553143f92 admin admin] PATH_INFO: `/v3/projects/a88297f8e3b54a5586d51a3362c84fbd/users/4b92a63daa6f488d914ed10ba2f5baf6/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.564130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e3302-6ef4-40e5-b206-08e553143f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88297f8e3b54a5586d51a3362c84fbd, user_id=4b92a63daa6f488d914ed10ba2f5baf6, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.566821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e3302-6ef4-40e5-b206-08e553143f92 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.584440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1345/2700] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:04 2022] PUT /identity/v3/projects/a88297f8e3b54a5586d51a3362c84fbd/users/4b92a63daa6f488d914ed10ba2f5baf6/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:46:04.589002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69f659-120c-45ca-8c8e-370246d09de5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.601037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69f659-120c-45ca-8c8e-370246d09de5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.610964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:04.671800 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888cbd4-a2c4-4d12-873c-792e1641ac08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.672593 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888cbd4-a2c4-4d12-873c-792e1641ac08 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.672875 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888cbd4-a2c4-4d12-873c-792e1641ac08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.673153 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888cbd4-a2c4-4d12-873c-792e1641ac08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.673884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888cbd4-a2c4-4d12-873c-792e1641ac08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.675633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888cbd4-a2c4-4d12-873c-792e1641ac08 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.687781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1356/2701] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:04.691785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e3302-6ef4-40e5-b206-08e553143f92 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.703162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e3302-6ef4-40e5-b206-08e553143f92 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.709134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcda8b0-e76c-4424-b97a-f89109c22086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.710052 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcda8b0-e76c-4424-b97a-f89109c22086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.710335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcda8b0-e76c-4424-b97a-f89109c22086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.710614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcda8b0-e76c-4424-b97a-f89109c22086 admin admin] PATH_INFO: `/v3/projects/a88297f8e3b54a5586d51a3362c84fbd/users/4b92a63daa6f488d914ed10ba2f5baf6/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.712635 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcda8b0-e76c-4424-b97a-f89109c22086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88297f8e3b54a5586d51a3362c84fbd, user_id=4b92a63daa6f488d914ed10ba2f5baf6, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.715528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcda8b0-e76c-4424-b97a-f89109c22086 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.726939 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1346/2702] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:04 2022] PUT /identity/v3/projects/a88297f8e3b54a5586d51a3362c84fbd/users/4b92a63daa6f488d914ed10ba2f5baf6/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:46:04.737625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90675a8-f298-4a87-a399-26c88a7ff64e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.737912 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90675a8-f298-4a87-a399-26c88a7ff64e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.738188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90675a8-f298-4a87-a399-26c88a7ff64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.767215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b90675a8-f298-4a87-a399-26c88a7ff64e None None] MFA Rules not processed for user `4b92a63daa6f488d914ed10ba2f5baf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:04.768498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b90675a8-f298-4a87-a399-26c88a7ff64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:04.842073 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1357/2703] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:04.852906 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcda8b0-e76c-4424-b97a-f89109c22086 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:04.854780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-8fcda8b0-e76c-4424-b97a-f89109c22086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:04.903271 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:04.916535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcda8b0-e76c-4424-b97a-f89109c22086 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:04.920686 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3e266-1321-4fb7-b6f8-b4fcba55b3b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:04.921346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3e266-1321-4fb7-b6f8-b4fcba55b3b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:04.921650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3e266-1321-4fb7-b6f8-b4fcba55b3b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:04.921934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3e266-1321-4fb7-b6f8-b4fcba55b3b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:04.931348 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3e266-1321-4fb7-b6f8-b4fcba55b3b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:04.932809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3e266-1321-4fb7-b6f8-b4fcba55b3b6 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:04.942843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1347/2704] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:05.046273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90675a8-f298-4a87-a399-26c88a7ff64e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:05.057504 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90675a8-f298-4a87-a399-26c88a7ff64e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:05.061302 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895a6a3-5270-4d3a-b006-4c10fbfcd147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:05.061963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895a6a3-5270-4d3a-b006-4c10fbfcd147 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:05.062247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895a6a3-5270-4d3a-b006-4c10fbfcd147 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:05.062495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895a6a3-5270-4d3a-b006-4c10fbfcd147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:05.123886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895a6a3-5270-4d3a-b006-4c10fbfcd147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:05.125341 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895a6a3-5270-4d3a-b006-4c10fbfcd147 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:05.135771 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1358/2705] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:07.749016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3e266-1321-4fb7-b6f8-b4fcba55b3b6 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:07.816979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3e266-1321-4fb7-b6f8-b4fcba55b3b6 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:07.822872 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:07.823832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:07.824101 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:07.824368 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] PATH_INFO: `/v3/users/5753a340076847828322c3d9e84e16fe` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:07.825713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5753a340076847828322c3d9e84e16fe)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:07.830942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:07.885444 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5753a340076847828322c3d9e84e16fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:07.886114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5753a340076847828322c3d9e84e16fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:07.895691 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5753a340076847828322c3d9e84e16fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:07.896484 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5753a340076847828322c3d9e84e16fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:07.909294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5753a340076847828322c3d9e84e16fe'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:07.922077 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1348/2706] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:07 2022] DELETE /identity/v3/users/5753a340076847828322c3d9e84e16fe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:07.927123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b9b4-99f3-43a4-91cc-98093675d323 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:07.927683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b9b4-99f3-43a4-91cc-98093675d323 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:07.927683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8b9b4-99f3-43a4-91cc-98093675d323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:07.948801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-51e8b9b4-99f3-43a4-91cc-98093675d323 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:07.950034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-51e8b9b4-99f3-43a4-91cc-98093675d323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:07.978342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:08.012835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1359/2707] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:08.023134 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:08.024826 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:08.083043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94464be2-b4b9-44d2-b24b-bb3d457719f9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:08.087643 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0271b-722f-4958-813b-70eaa0b0a9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:08.088279 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0271b-722f-4958-813b-70eaa0b0a9bb service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:08.088559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0271b-722f-4958-813b-70eaa0b0a9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:08.088831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0271b-722f-4958-813b-70eaa0b0a9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:08.103260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0271b-722f-4958-813b-70eaa0b0a9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:08.105262 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0271b-722f-4958-813b-70eaa0b0a9bb service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:08.117864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1349/2708] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:08.674494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b9b4-99f3-43a4-91cc-98093675d323 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:08.676069 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-51e8b9b4-99f3-43a4-91cc-98093675d323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:08.694497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8b9b4-99f3-43a4-91cc-98093675d323 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:08.699435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:08.700333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:08.700650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:08.700938 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] PATH_INFO: `/v3/projects/7f958221d01d48ff9955895e76825fc0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:08.702368 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f958221d01d48ff9955895e76825fc0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:08.704411 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:08.754459 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f958221d01d48ff9955895e76825fc0', 'request_id': 'req-3b3a2e85-901c-4750-93b3-1e362e424814'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.759928 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f958221d01d48ff9955895e76825fc0', 'request_id': 'req-3b3a2e85-901c-4750-93b3-1e362e424814'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.766175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f958221d01d48ff9955895e76825fc0', 'request_id': 'req-3b3a2e85-901c-4750-93b3-1e362e424814'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.780502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] The token cache is being invalidate because project 7f958221d01d48ff9955895e76825fc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:08.780789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.781458 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.783374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1360/2709] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:08 2022] DELETE /identity/v3/projects/7f958221d01d48ff9955895e76825fc0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:08.787427 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0271b-722f-4958-813b-70eaa0b0a9bb service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:08.798209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0271b-722f-4958-813b-70eaa0b0a9bb service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:08.802478 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:08.860165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:08.861039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:08.861317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:08.861589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] PATH_INFO: `/v3/users/4b92a63daa6f488d914ed10ba2f5baf6` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:08.862795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b92a63daa6f488d914ed10ba2f5baf6)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:08.864780 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:08.914923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b92a63daa6f488d914ed10ba2f5baf6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.915769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b92a63daa6f488d914ed10ba2f5baf6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.924863 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b92a63daa6f488d914ed10ba2f5baf6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.925498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b92a63daa6f488d914ed10ba2f5baf6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.938728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-7d87a580-4902-4cae-8756-4aa2df4a8b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b92a63daa6f488d914ed10ba2f5baf6'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:08.950491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1350/2710] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:08 2022] DELETE /identity/v3/users/4b92a63daa6f488d914ed10ba2f5baf6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:09.247276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:09.248853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:09.265254 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a2e85-901c-4750-93b3-1e362e424814 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:09.325066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:09.326363 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.327606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.328591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] PATH_INFO: `/v3/projects/a88297f8e3b54a5586d51a3362c84fbd` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.330140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a88297f8e3b54a5586d51a3362c84fbd)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:09.332953 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:09.385001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a88297f8e3b54a5586d51a3362c84fbd', 'request_id': 'req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:09.390144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:09.391779 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a88297f8e3b54a5586d51a3362c84fbd', 'request_id': 'req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:09.400025 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a88297f8e3b54a5586d51a3362c84fbd', 'request_id': 'req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:09.415438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] The token cache is being invalidate because project a88297f8e3b54a5586d51a3362c84fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:09.415724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:09.416424 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-44a7b070-97a6-49e0-b374-5c7f5ed6d8f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:09.418432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1361/2711] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:09 2022] DELETE /identity/v3/projects/a88297f8e3b54a5586d51a3362c84fbd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:09.423529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f89b7-c960-4905-bf2a-7eb3d80bc289 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.423819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f89b7-c960-4905-bf2a-7eb3d80bc289 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.424094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f89b7-c960-4905-bf2a-7eb3d80bc289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.442654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-a63f89b7-c960-4905-bf2a-7eb3d80bc289 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:09.443918 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-a63f89b7-c960-4905-bf2a-7eb3d80bc289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:09.514168 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1351/2712] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:09.523592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54757c-24d1-440f-9601-45513998536f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.523877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54757c-24d1-440f-9601-45513998536f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.524154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54757c-24d1-440f-9601-45513998536f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.543546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6f54757c-24d1-440f-9601-45513998536f None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:09.544744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6f54757c-24d1-440f-9601-45513998536f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:09.553457 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1362/2713] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:09.558025 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f89b7-c960-4905-bf2a-7eb3d80bc289 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:09.568559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f89b7-c960-4905-bf2a-7eb3d80bc289 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:09.574475 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f184731-c914-47e2-ba36-108c4da5fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:09.575202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f184731-c914-47e2-ba36-108c4da5fb8f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.575453 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f184731-c914-47e2-ba36-108c4da5fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.575724 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f184731-c914-47e2-ba36-108c4da5fb8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.576458 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f184731-c914-47e2-ba36-108c4da5fb8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:46:09.576765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f184731-c914-47e2-ba36-108c4da5fb8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:09.578383 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f184731-c914-47e2-ba36-108c4da5fb8f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:09.591328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1352/2714] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:46:09.595867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54757c-24d1-440f-9601-45513998536f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:09.605656 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54757c-24d1-440f-9601-45513998536f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:09.611655 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ae1db-b66e-472a-a3d9-1063f88999cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:09.612317 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ae1db-b66e-472a-a3d9-1063f88999cc admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.612602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ae1db-b66e-472a-a3d9-1063f88999cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.612879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ae1db-b66e-472a-a3d9-1063f88999cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.613571 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ae1db-b66e-472a-a3d9-1063f88999cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:09.615528 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ae1db-b66e-472a-a3d9-1063f88999cc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:09.647637 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1363/2715] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:09 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:09.652535 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f184731-c914-47e2-ba36-108c4da5fb8f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:09.664582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f184731-c914-47e2-ba36-108c4da5fb8f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:09.674321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:09.729548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4fab7-8841-4814-b437-30215b75c988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:09.730220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4fab7-8841-4814-b437-30215b75c988 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.730499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4fab7-8841-4814-b437-30215b75c988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.730772 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4fab7-8841-4814-b437-30215b75c988 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.731512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4fab7-8841-4814-b437-30215b75c988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:09.733451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b4fab7-8841-4814-b437-30215b75c988 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:09.756441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1353/2716] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:09 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:09.760871 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ae1db-b66e-472a-a3d9-1063f88999cc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:09.771873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ae1db-b66e-472a-a3d9-1063f88999cc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:09.776939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8ee43-c8f0-4552-ad13-625e488ca1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:09.777617 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ee43-c8f0-4552-ad13-625e488ca1ee admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.777887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ee43-c8f0-4552-ad13-625e488ca1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.778164 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ee43-c8f0-4552-ad13-625e488ca1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.778916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8ee43-c8f0-4552-ad13-625e488ca1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:09.780885 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8ee43-c8f0-4552-ad13-625e488ca1ee admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:09.794141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1364/2717] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:09.798681 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4fab7-8841-4814-b437-30215b75c988 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:09.812189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b4fab7-8841-4814-b437-30215b75c988 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:09.817428 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52494e-83cf-4ffc-b799-9509389b49db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:09.818085 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52494e-83cf-4ffc-b799-9509389b49db admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.818365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52494e-83cf-4ffc-b799-9509389b49db admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.818636 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52494e-83cf-4ffc-b799-9509389b49db admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.819343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52494e-83cf-4ffc-b799-9509389b49db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:09.820967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52494e-83cf-4ffc-b799-9509389b49db admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:09.831440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1354/2718] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:09.835683 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8ee43-c8f0-4552-ad13-625e488ca1ee admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:09.845925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8ee43-c8f0-4552-ad13-625e488ca1ee admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:09.850970 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df2783-a9a3-464a-a4fd-fdf22b342d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:09.851890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df2783-a9a3-464a-a4fd-fdf22b342d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.852171 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df2783-a9a3-464a-a4fd-fdf22b342d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.852444 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df2783-a9a3-464a-a4fd-fdf22b342d3b admin admin] PATH_INFO: `/v3/projects/8aaf2758774b40d4973e54e7284285f2/users/17a2508405514afda746feffe9c12d6e/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.868183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df2783-a9a3-464a-a4fd-fdf22b342d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aaf2758774b40d4973e54e7284285f2, user_id=17a2508405514afda746feffe9c12d6e, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:09.870868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df2783-a9a3-464a-a4fd-fdf22b342d3b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:09.885588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1365/2719] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:09 2022] PUT /identity/v3/projects/8aaf2758774b40d4973e54e7284285f2/users/17a2508405514afda746feffe9c12d6e/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:09.896127 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dfc72-facd-4cc4-ade4-20c0db206d7c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:09.896415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dfc72-facd-4cc4-ade4-20c0db206d7c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:09.896690 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dfc72-facd-4cc4-ade4-20c0db206d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:09.924335 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-197dfc72-facd-4cc4-ade4-20c0db206d7c None None] MFA Rules not processed for user `17a2508405514afda746feffe9c12d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:09.925557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-197dfc72-facd-4cc4-ade4-20c0db206d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:09.991519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1355/2720] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:09.996133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df2783-a9a3-464a-a4fd-fdf22b342d3b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:10.061476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df2783-a9a3-464a-a4fd-fdf22b342d3b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:10.067060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121b1f0-a17d-414e-96b2-ef45a6df4c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:10.067786 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121b1f0-a17d-414e-96b2-ef45a6df4c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.068049 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121b1f0-a17d-414e-96b2-ef45a6df4c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.068339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121b1f0-a17d-414e-96b2-ef45a6df4c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.069035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121b1f0-a17d-414e-96b2-ef45a6df4c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:10.070964 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121b1f0-a17d-414e-96b2-ef45a6df4c9d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:10.102392 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1366/2721] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:09 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:10.106639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dfc72-facd-4cc4-ade4-20c0db206d7c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:10.121507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dfc72-facd-4cc4-ade4-20c0db206d7c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:10.180604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ca0c4-ef5b-4818-9d44-7420bea58efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:10.181280 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ca0c4-ef5b-4818-9d44-7420bea58efa admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.181559 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ca0c4-ef5b-4818-9d44-7420bea58efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.181816 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ca0c4-ef5b-4818-9d44-7420bea58efa admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.182511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ca0c4-ef5b-4818-9d44-7420bea58efa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:10.184484 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ca0c4-ef5b-4818-9d44-7420bea58efa admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:10.206545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1356/2722] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:10 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:10.210588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121b1f0-a17d-414e-96b2-ef45a6df4c9d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:10.221481 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121b1f0-a17d-414e-96b2-ef45a6df4c9d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:10.226140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c21bab-d6a3-48a5-95b9-f9ef924050fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:10.226831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c21bab-d6a3-48a5-95b9-f9ef924050fd admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.227107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c21bab-d6a3-48a5-95b9-f9ef924050fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.227422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c21bab-d6a3-48a5-95b9-f9ef924050fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.228147 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c21bab-d6a3-48a5-95b9-f9ef924050fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:10.229757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c21bab-d6a3-48a5-95b9-f9ef924050fd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:10.241616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1367/2723] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:10.245819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ca0c4-ef5b-4818-9d44-7420bea58efa admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:10.256513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ca0c4-ef5b-4818-9d44-7420bea58efa admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:10.262646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64019a7-9ac8-41ea-97f5-a8ebd539d486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:10.263573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64019a7-9ac8-41ea-97f5-a8ebd539d486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.263853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64019a7-9ac8-41ea-97f5-a8ebd539d486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.264125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64019a7-9ac8-41ea-97f5-a8ebd539d486 admin admin] PATH_INFO: `/v3/projects/fe4e84c2eccd4246b7fadf71b7665c1b/users/91256ff75cff462c8612b6d4e2eddadb/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.273108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:10.281639 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64019a7-9ac8-41ea-97f5-a8ebd539d486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe4e84c2eccd4246b7fadf71b7665c1b, user_id=91256ff75cff462c8612b6d4e2eddadb, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:10.284464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64019a7-9ac8-41ea-97f5-a8ebd539d486 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:10.302039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1357/2724] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:10 2022] PUT /identity/v3/projects/fe4e84c2eccd4246b7fadf71b7665c1b/users/91256ff75cff462c8612b6d4e2eddadb/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:10.306041 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c21bab-d6a3-48a5-95b9-f9ef924050fd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:10.316670 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c21bab-d6a3-48a5-95b9-f9ef924050fd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:10.369143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:10.378187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e017d-7e12-45b7-a8d7-baa6d4710499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:10.378891 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e017d-7e12-45b7-a8d7-baa6d4710499 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.379191 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e017d-7e12-45b7-a8d7-baa6d4710499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.379539 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e017d-7e12-45b7-a8d7-baa6d4710499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.380285 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e017d-7e12-45b7-a8d7-baa6d4710499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:10.382020 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e017d-7e12-45b7-a8d7-baa6d4710499 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:10.392538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1368/2725] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:10.396956 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64019a7-9ac8-41ea-97f5-a8ebd539d486 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:10.409031 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64019a7-9ac8-41ea-97f5-a8ebd539d486 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:10.413937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f9e11-9049-4a92-9148-3f2f9c2b4e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:10.414827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f9e11-9049-4a92-9148-3f2f9c2b4e2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.415112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f9e11-9049-4a92-9148-3f2f9c2b4e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.415424 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f9e11-9049-4a92-9148-3f2f9c2b4e2b admin admin] PATH_INFO: `/v3/projects/fe4e84c2eccd4246b7fadf71b7665c1b/users/91256ff75cff462c8612b6d4e2eddadb/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.417774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f9e11-9049-4a92-9148-3f2f9c2b4e2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe4e84c2eccd4246b7fadf71b7665c1b, user_id=91256ff75cff462c8612b6d4e2eddadb, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:10.420497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f9e11-9049-4a92-9148-3f2f9c2b4e2b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:10.431710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1358/2726] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:10 2022] PUT /identity/v3/projects/fe4e84c2eccd4246b7fadf71b7665c1b/users/91256ff75cff462c8612b6d4e2eddadb/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:10.448743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90848fe8-4fd0-4f5f-8ce2-9d7a9375255e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.449046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90848fe8-4fd0-4f5f-8ce2-9d7a9375255e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.449384 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90848fe8-4fd0-4f5f-8ce2-9d7a9375255e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.479540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-90848fe8-4fd0-4f5f-8ce2-9d7a9375255e None None] MFA Rules not processed for user `91256ff75cff462c8612b6d4e2eddadb`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:10.480755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-90848fe8-4fd0-4f5f-8ce2-9d7a9375255e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:10.549887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1369/2727] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:10.560887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f9e11-9049-4a92-9148-3f2f9c2b4e2b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:10.562703 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b06f9e11-9049-4a92-9148-3f2f9c2b4e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:10.621382 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f9e11-9049-4a92-9148-3f2f9c2b4e2b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:10.625390 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb1518-76b7-4c4f-94d8-de267858e436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:10.626043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1518-76b7-4c4f-94d8-de267858e436 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.626332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1518-76b7-4c4f-94d8-de267858e436 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.626596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb1518-76b7-4c4f-94d8-de267858e436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.635878 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb1518-76b7-4c4f-94d8-de267858e436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:10.637338 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb1518-76b7-4c4f-94d8-de267858e436 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:10.646876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1359/2728] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:10.779711 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90848fe8-4fd0-4f5f-8ce2-9d7a9375255e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:10.852671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90848fe8-4fd0-4f5f-8ce2-9d7a9375255e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:10.858498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:10.859447 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.859727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.860319 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] PATH_INFO: `/v3/users/17a2508405514afda746feffe9c12d6e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.861530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17a2508405514afda746feffe9c12d6e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:10.863536 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:10.918068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17a2508405514afda746feffe9c12d6e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:10.918741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17a2508405514afda746feffe9c12d6e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:10.927899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17a2508405514afda746feffe9c12d6e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:10.942371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17a2508405514afda746feffe9c12d6e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:10.943353 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17a2508405514afda746feffe9c12d6e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:10.950679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:10.960331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1370/2729] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:10 2022] DELETE /identity/v3/users/17a2508405514afda746feffe9c12d6e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:10.965819 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942c2af-4982-4ed2-a0c9-02d047c24cb0 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:10.966636 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942c2af-4982-4ed2-a0c9-02d047c24cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:10.966809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942c2af-4982-4ed2-a0c9-02d047c24cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:10.986430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b942c2af-4982-4ed2-a0c9-02d047c24cb0 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:10.987655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b942c2af-4982-4ed2-a0c9-02d047c24cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:11.049102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1360/2730] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:46:11.059043 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:11.060704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:11.126974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86c824-73cb-4eca-a851-9acc0802f9f8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:11.131250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268db29b-a4e6-47cf-8105-6dbaba6c5006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:11.131944 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268db29b-a4e6-47cf-8105-6dbaba6c5006 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:11.132224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268db29b-a4e6-47cf-8105-6dbaba6c5006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:11.132496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268db29b-a4e6-47cf-8105-6dbaba6c5006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:11.142733 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268db29b-a4e6-47cf-8105-6dbaba6c5006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:11.144214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268db29b-a4e6-47cf-8105-6dbaba6c5006 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:11.154003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1371/2731] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:11.445068 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942c2af-4982-4ed2-a0c9-02d047c24cb0 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:11.446677 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b942c2af-4982-4ed2-a0c9-02d047c24cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:11.459500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942c2af-4982-4ed2-a0c9-02d047c24cb0 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:11.464614 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:11.465590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:11.465947 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:11.466296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] PATH_INFO: `/v3/projects/8aaf2758774b40d4973e54e7284285f2` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:11.467672 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aaf2758774b40d4973e54e7284285f2)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:11.469575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:11.521209 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aaf2758774b40d4973e54e7284285f2', 'request_id': 'req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.530849 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aaf2758774b40d4973e54e7284285f2', 'request_id': 'req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.537758 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aaf2758774b40d4973e54e7284285f2', 'request_id': 'req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.546117 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:11.550671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] The token cache is being invalidate because project 8aaf2758774b40d4973e54e7284285f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:11.550962 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.551853 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.553687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1361/2732] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:11 2022] DELETE /identity/v3/projects/8aaf2758774b40d4973e54e7284285f2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:11.557899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268db29b-a4e6-47cf-8105-6dbaba6c5006 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:11.568126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268db29b-a4e6-47cf-8105-6dbaba6c5006 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:11.638541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:11.639479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:11.639757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:11.640032 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] PATH_INFO: `/v3/users/91256ff75cff462c8612b6d4e2eddadb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:11.641502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=91256ff75cff462c8612b6d4e2eddadb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:11.643512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:11.695631 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91256ff75cff462c8612b6d4e2eddadb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.696197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91256ff75cff462c8612b6d4e2eddadb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.704326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91256ff75cff462c8612b6d4e2eddadb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.716241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91256ff75cff462c8612b6d4e2eddadb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.716957 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4915bca4-8766-4cbc-b72e-2a0f99da2c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91256ff75cff462c8612b6d4e2eddadb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:11.730145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1372/2733] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:11 2022] DELETE /identity/v3/users/91256ff75cff462c8612b6d4e2eddadb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:46:12.036623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:12.038102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:12.052710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda0e2c-80a5-4ab8-89a3-3bfd1f1b7ba7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:12.113580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:12.114456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.114736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.115009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] PATH_INFO: `/v3/projects/fe4e84c2eccd4246b7fadf71b7665c1b` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.116339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe4e84c2eccd4246b7fadf71b7665c1b)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:12.118268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:12.169138 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe4e84c2eccd4246b7fadf71b7665c1b', 'request_id': 'req-4d77ec2d-67d6-4db2-9c94-90c11c95488f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:12.178372 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe4e84c2eccd4246b7fadf71b7665c1b', 'request_id': 'req-4d77ec2d-67d6-4db2-9c94-90c11c95488f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:12.184805 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe4e84c2eccd4246b7fadf71b7665c1b', 'request_id': 'req-4d77ec2d-67d6-4db2-9c94-90c11c95488f'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:12.196434 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] The token cache is being invalidate because project fe4e84c2eccd4246b7fadf71b7665c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:12.196728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:12.197470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4d77ec2d-67d6-4db2-9c94-90c11c95488f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:12.199179 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1362/2734] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:12 2022] DELETE /identity/v3/projects/fe4e84c2eccd4246b7fadf71b7665c1b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:46:12.204192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb96a7-bc11-43e6-814f-f0809d66efa9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.204478 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb96a7-bc11-43e6-814f-f0809d66efa9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.204760 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb96a7-bc11-43e6-814f-f0809d66efa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.223523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-63eb96a7-bc11-43e6-814f-f0809d66efa9 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:12.224742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-63eb96a7-bc11-43e6-814f-f0809d66efa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:12.238577 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:12.288007 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1373/2735] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:12.296518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7bd30-b5de-4f01-8f4b-177929b81399 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.296828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7bd30-b5de-4f01-8f4b-177929b81399 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.297106 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7bd30-b5de-4f01-8f4b-177929b81399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.316777 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-1bc7bd30-b5de-4f01-8f4b-177929b81399 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:12.317985 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-1bc7bd30-b5de-4f01-8f4b-177929b81399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:12.326394 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1363/2736] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:12.330883 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb96a7-bc11-43e6-814f-f0809d66efa9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:12.340745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb96a7-bc11-43e6-814f-f0809d66efa9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:12.345554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5b3e6-11e5-4bc2-a607-a6ea7d9eab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:12.346204 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5b3e6-11e5-4bc2-a607-a6ea7d9eab80 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.346483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5b3e6-11e5-4bc2-a607-a6ea7d9eab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.346749 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5b3e6-11e5-4bc2-a607-a6ea7d9eab80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.347541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5b3e6-11e5-4bc2-a607-a6ea7d9eab80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:46:12.347868 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5b3e6-11e5-4bc2-a607-a6ea7d9eab80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:12.349437 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5b3e6-11e5-4bc2-a607-a6ea7d9eab80 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:12.361214 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1374/2737] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:46:12.366392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7bd30-b5de-4f01-8f4b-177929b81399 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:12.379654 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7bd30-b5de-4f01-8f4b-177929b81399 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:12.385328 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce5683-2280-4474-9416-e3f982710263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:12.385996 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce5683-2280-4474-9416-e3f982710263 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.386282 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce5683-2280-4474-9416-e3f982710263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.386529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce5683-2280-4474-9416-e3f982710263 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.387218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce5683-2280-4474-9416-e3f982710263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:12.389255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce5683-2280-4474-9416-e3f982710263 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:12.418586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1364/2738] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:12 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:12.424332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5b3e6-11e5-4bc2-a607-a6ea7d9eab80 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:12.435609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5b3e6-11e5-4bc2-a607-a6ea7d9eab80 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:12.495107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67dc95-2fdb-4758-9161-04832a1dbe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:12.495837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67dc95-2fdb-4758-9161-04832a1dbe9b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.496119 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67dc95-2fdb-4758-9161-04832a1dbe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.496397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67dc95-2fdb-4758-9161-04832a1dbe9b admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.497091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67dc95-2fdb-4758-9161-04832a1dbe9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:12.499017 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67dc95-2fdb-4758-9161-04832a1dbe9b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:12.520205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1375/2739] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:12 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:12.525830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce5683-2280-4474-9416-e3f982710263 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:12.536793 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce5683-2280-4474-9416-e3f982710263 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:12.542394 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06143e16-e34c-4ada-8465-1dea6566a941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:12.543071 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06143e16-e34c-4ada-8465-1dea6566a941 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.543325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06143e16-e34c-4ada-8465-1dea6566a941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.543625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06143e16-e34c-4ada-8465-1dea6566a941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.544333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06143e16-e34c-4ada-8465-1dea6566a941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:12.545973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06143e16-e34c-4ada-8465-1dea6566a941 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:12.556909 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1365/2740] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:12.561216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67dc95-2fdb-4758-9161-04832a1dbe9b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:12.571981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67dc95-2fdb-4758-9161-04832a1dbe9b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:12.577839 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548ec07-e6cb-4d04-b460-4b2f58c56bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:12.578697 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548ec07-e6cb-4d04-b460-4b2f58c56bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.578863 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548ec07-e6cb-4d04-b460-4b2f58c56bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.579104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548ec07-e6cb-4d04-b460-4b2f58c56bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.579847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548ec07-e6cb-4d04-b460-4b2f58c56bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:12.581455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548ec07-e6cb-4d04-b460-4b2f58c56bcd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:12.593976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1376/2741] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:12.598463 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06143e16-e34c-4ada-8465-1dea6566a941 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:12.609002 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06143e16-e34c-4ada-8465-1dea6566a941 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:12.614069 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3c873-fe3b-4646-86aa-a7084408833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:12.614945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3c873-fe3b-4646-86aa-a7084408833e admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.615250 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3c873-fe3b-4646-86aa-a7084408833e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.615553 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3c873-fe3b-4646-86aa-a7084408833e admin admin] PATH_INFO: `/v3/projects/548065eb05d44d74b3710d9c2e95a828/users/410b0c302ccc41fc98a8552d433e23fc/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.630875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3c873-fe3b-4646-86aa-a7084408833e admin admin] RBAC: Authorizing `identity:create_grant(project_id=548065eb05d44d74b3710d9c2e95a828, user_id=410b0c302ccc41fc98a8552d433e23fc, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:12.633557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3c873-fe3b-4646-86aa-a7084408833e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:12.648837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1366/2742] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:12 2022] PUT /identity/v3/projects/548065eb05d44d74b3710d9c2e95a828/users/410b0c302ccc41fc98a8552d433e23fc/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:12.659349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f264-499b-45b1-8122-bbc731ee0400 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.659618 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f264-499b-45b1-8122-bbc731ee0400 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.659890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f264-499b-45b1-8122-bbc731ee0400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.692282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2674f264-499b-45b1-8122-bbc731ee0400 None None] MFA Rules not processed for user `410b0c302ccc41fc98a8552d433e23fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:12.693998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2674f264-499b-45b1-8122-bbc731ee0400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:12.768202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1377/2743] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:12.772766 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3c873-fe3b-4646-86aa-a7084408833e admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:12.804589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:12.841133 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3c873-fe3b-4646-86aa-a7084408833e admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:12.847035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c63d9-8a9b-4d31-a837-3bd4e84ae65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:12.847710 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c63d9-8a9b-4d31-a837-3bd4e84ae65f admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.847990 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c63d9-8a9b-4d31-a837-3bd4e84ae65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.848264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c63d9-8a9b-4d31-a837-3bd4e84ae65f admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.848982 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c63d9-8a9b-4d31-a837-3bd4e84ae65f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:12.850923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c63d9-8a9b-4d31-a837-3bd4e84ae65f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:12.884995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1367/2744] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:12 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:12.889388 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674f264-499b-45b1-8122-bbc731ee0400 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:12.901407 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674f264-499b-45b1-8122-bbc731ee0400 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:12.911503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:12.962391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c116694-5d63-42d4-88a7-9d6d7325d4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:12.963064 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c116694-5d63-42d4-88a7-9d6d7325d4ff admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:12.963380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c116694-5d63-42d4-88a7-9d6d7325d4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:12.963658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c116694-5d63-42d4-88a7-9d6d7325d4ff admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:12.964349 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c116694-5d63-42d4-88a7-9d6d7325d4ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:12.966288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c116694-5d63-42d4-88a7-9d6d7325d4ff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:12.991339 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1378/2745] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:12 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:12.996582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c63d9-8a9b-4d31-a837-3bd4e84ae65f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:13.007583 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c63d9-8a9b-4d31-a837-3bd4e84ae65f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:13.012492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298784b-4c6f-4767-9f8c-afe39abec79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:13.013154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298784b-4c6f-4767-9f8c-afe39abec79c admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:13.013438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298784b-4c6f-4767-9f8c-afe39abec79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:13.013720 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298784b-4c6f-4767-9f8c-afe39abec79c admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:13.014441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298784b-4c6f-4767-9f8c-afe39abec79c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:13.016093 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298784b-4c6f-4767-9f8c-afe39abec79c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:13.027455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1368/2746] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:13.031494 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c116694-5d63-42d4-88a7-9d6d7325d4ff admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:13.042471 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c116694-5d63-42d4-88a7-9d6d7325d4ff admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:13.047646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4149d9-c20d-491e-a26a-560fc99c6ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:13.048579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4149d9-c20d-491e-a26a-560fc99c6ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:13.048865 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4149d9-c20d-491e-a26a-560fc99c6ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:13.049139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4149d9-c20d-491e-a26a-560fc99c6ca1 admin admin] PATH_INFO: `/v3/projects/9e573d8f2bd34313a42dd57a3730e5f8/users/3e2fe4acc24049eca004e2fff9d02c83/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:13.063956 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4149d9-c20d-491e-a26a-560fc99c6ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e573d8f2bd34313a42dd57a3730e5f8, user_id=3e2fe4acc24049eca004e2fff9d02c83, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:13.066979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4149d9-c20d-491e-a26a-560fc99c6ca1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:13.083237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1379/2747] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:13 2022] PUT /identity/v3/projects/9e573d8f2bd34313a42dd57a3730e5f8/users/3e2fe4acc24049eca004e2fff9d02c83/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:13.088340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298784b-4c6f-4767-9f8c-afe39abec79c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:13.102505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298784b-4c6f-4767-9f8c-afe39abec79c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:13.160255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cf50e-4660-47fa-8936-c821e0438bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:13.160931 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cf50e-4660-47fa-8936-c821e0438bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:13.161220 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cf50e-4660-47fa-8936-c821e0438bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:13.161497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cf50e-4660-47fa-8936-c821e0438bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:13.162225 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cf50e-4660-47fa-8936-c821e0438bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:13.163882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cf50e-4660-47fa-8936-c821e0438bd2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:13.174678 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1369/2748] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:13.179122 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4149d9-c20d-491e-a26a-560fc99c6ca1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:13.190546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4149d9-c20d-491e-a26a-560fc99c6ca1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:13.195590 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c3ec0-87c7-4a51-b254-58705282acd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:13.196530 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c3ec0-87c7-4a51-b254-58705282acd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:13.196835 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c3ec0-87c7-4a51-b254-58705282acd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:13.197114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c3ec0-87c7-4a51-b254-58705282acd7 admin admin] PATH_INFO: `/v3/projects/9e573d8f2bd34313a42dd57a3730e5f8/users/3e2fe4acc24049eca004e2fff9d02c83/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:13.199126 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c3ec0-87c7-4a51-b254-58705282acd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e573d8f2bd34313a42dd57a3730e5f8, user_id=3e2fe4acc24049eca004e2fff9d02c83, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:13.201809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c3ec0-87c7-4a51-b254-58705282acd7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:13.212674 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1380/2749] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:13 2022] PUT /identity/v3/projects/9e573d8f2bd34313a42dd57a3730e5f8/users/3e2fe4acc24049eca004e2fff9d02c83/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:13.224454 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9269274-c179-4669-98f9-e142d6e23462 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:13.224745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9269274-c179-4669-98f9-e142d6e23462 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:13.225022 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9269274-c179-4669-98f9-e142d6e23462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:13.252491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-b9269274-c179-4669-98f9-e142d6e23462 None None] MFA Rules not processed for user `3e2fe4acc24049eca004e2fff9d02c83`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:13.253727 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-b9269274-c179-4669-98f9-e142d6e23462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:13.323629 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1370/2750] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:13.334804 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c3ec0-87c7-4a51-b254-58705282acd7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:13.336695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cf2c3ec0-87c7-4a51-b254-58705282acd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:13.396169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c3ec0-87c7-4a51-b254-58705282acd7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:13.400200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd98a3-f2ab-439e-8d8d-cb8895b0f4c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:13.400854 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd98a3-f2ab-439e-8d8d-cb8895b0f4c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:13.401133 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd98a3-f2ab-439e-8d8d-cb8895b0f4c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:13.401406 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd98a3-f2ab-439e-8d8d-cb8895b0f4c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:13.411744 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd98a3-f2ab-439e-8d8d-cb8895b0f4c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:13.413229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd98a3-f2ab-439e-8d8d-cb8895b0f4c9 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:13.423344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1381/2751] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:14.001103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9269274-c179-4669-98f9-e142d6e23462 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:14.025734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:14.065288 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9269274-c179-4669-98f9-e142d6e23462 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:14.070645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:14.071550 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:14.071837 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:14.072114 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] PATH_INFO: `/v3/users/410b0c302ccc41fc98a8552d433e23fc` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:14.073379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=410b0c302ccc41fc98a8552d433e23fc)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:14.075343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:14.125546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '410b0c302ccc41fc98a8552d433e23fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.126191 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '410b0c302ccc41fc98a8552d433e23fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.137868 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '410b0c302ccc41fc98a8552d433e23fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.138587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '410b0c302ccc41fc98a8552d433e23fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.151043 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '410b0c302ccc41fc98a8552d433e23fc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.163217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1371/2752] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:13 2022] DELETE /identity/v3/users/410b0c302ccc41fc98a8552d433e23fc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:14.168404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7734d3-adbc-4ac6-a9d1-e1596dabec11 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:14.168876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7734d3-adbc-4ac6-a9d1-e1596dabec11 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:14.168985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7734d3-adbc-4ac6-a9d1-e1596dabec11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:14.187466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3b7734d3-adbc-4ac6-a9d1-e1596dabec11 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:14.188679 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3b7734d3-adbc-4ac6-a9d1-e1596dabec11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:14.255400 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1382/2753] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:14.266241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:14.268400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:14.329379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc76288-220e-4b03-a861-362665696d80 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:14.333457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e6b07-e385-4c5b-9ce7-5bf68c35346f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:14.334105 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e6b07-e385-4c5b-9ce7-5bf68c35346f service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:14.334388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e6b07-e385-4c5b-9ce7-5bf68c35346f service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:14.334664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e6b07-e385-4c5b-9ce7-5bf68c35346f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:14.344259 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e6b07-e385-4c5b-9ce7-5bf68c35346f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:14.345753 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e6b07-e385-4c5b-9ce7-5bf68c35346f service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:14.355949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1372/2754] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:14.632189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7734d3-adbc-4ac6-a9d1-e1596dabec11 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:14.633742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3b7734d3-adbc-4ac6-a9d1-e1596dabec11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:14.647053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7734d3-adbc-4ac6-a9d1-e1596dabec11 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:14.652422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:14.653502 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:14.653769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:14.654122 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] PATH_INFO: `/v3/projects/548065eb05d44d74b3710d9c2e95a828` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:14.655692 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=548065eb05d44d74b3710d9c2e95a828)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:14.657555 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:14.660240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:14.711120 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '548065eb05d44d74b3710d9c2e95a828', 'request_id': 'req-13d43dfe-58cd-45ef-b029-d11d88b739a5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.717008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '548065eb05d44d74b3710d9c2e95a828', 'request_id': 'req-13d43dfe-58cd-45ef-b029-d11d88b739a5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.723985 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '548065eb05d44d74b3710d9c2e95a828', 'request_id': 'req-13d43dfe-58cd-45ef-b029-d11d88b739a5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.738184 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] The token cache is being invalidate because project 548065eb05d44d74b3710d9c2e95a828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:14.738468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.739102 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.740729 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1383/2755] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:14 2022] DELETE /identity/v3/projects/548065eb05d44d74b3710d9c2e95a828 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:14.744795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e6b07-e385-4c5b-9ce7-5bf68c35346f service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:14.754887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e6b07-e385-4c5b-9ce7-5bf68c35346f service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:14.797462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:14.815769 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:14.816633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:14.816908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:14.817201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] PATH_INFO: `/v3/users/3e2fe4acc24049eca004e2fff9d02c83` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:14.818370 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2fe4acc24049eca004e2fff9d02c83)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:14.820340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:14.868764 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2fe4acc24049eca004e2fff9d02c83'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.869491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2fe4acc24049eca004e2fff9d02c83'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.879682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2fe4acc24049eca004e2fff9d02c83'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.880294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2fe4acc24049eca004e2fff9d02c83'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.893294 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d54d9329-eff2-4384-a23c-6977afc31de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2fe4acc24049eca004e2fff9d02c83'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:14.906506 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1373/2756] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:14 2022] DELETE /identity/v3/users/3e2fe4acc24049eca004e2fff9d02c83 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:15.197607 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:15.199088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:15.215198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d43dfe-58cd-45ef-b029-d11d88b739a5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:15.274645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:15.275550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.275828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.276099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] PATH_INFO: `/v3/projects/9e573d8f2bd34313a42dd57a3730e5f8` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.277287 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e573d8f2bd34313a42dd57a3730e5f8)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:15.279200 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:15.340924 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e573d8f2bd34313a42dd57a3730e5f8', 'request_id': 'req-234953b4-1ae7-4233-8b1c-c8d95246a078'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:15.347594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e573d8f2bd34313a42dd57a3730e5f8', 'request_id': 'req-234953b4-1ae7-4233-8b1c-c8d95246a078'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:15.353895 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e573d8f2bd34313a42dd57a3730e5f8', 'request_id': 'req-234953b4-1ae7-4233-8b1c-c8d95246a078'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:15.368316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] The token cache is being invalidate because project 9e573d8f2bd34313a42dd57a3730e5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:15.368603 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:15.369292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-234953b4-1ae7-4233-8b1c-c8d95246a078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:15.371028 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1384/2757] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:15 2022] DELETE /identity/v3/projects/9e573d8f2bd34313a42dd57a3730e5f8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:15.376206 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead0aab-b05f-4d29-917d-107defd6354d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.376785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead0aab-b05f-4d29-917d-107defd6354d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.376785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead0aab-b05f-4d29-917d-107defd6354d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.397560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-eead0aab-b05f-4d29-917d-107defd6354d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:15.399269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-eead0aab-b05f-4d29-917d-107defd6354d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:15.463504 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1374/2758] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:15.471687 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcf7ab-b3b3-4a2a-9dd0-eaec805a7607 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.471974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcf7ab-b3b3-4a2a-9dd0-eaec805a7607 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.472250 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcf7ab-b3b3-4a2a-9dd0-eaec805a7607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.491198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-02dcf7ab-b3b3-4a2a-9dd0-eaec805a7607 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:15.492432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-02dcf7ab-b3b3-4a2a-9dd0-eaec805a7607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:15.501720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1385/2759] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:46:15.506278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead0aab-b05f-4d29-917d-107defd6354d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:15.516130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead0aab-b05f-4d29-917d-107defd6354d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:15.521420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7f378-029f-4352-9e71-5746991a03f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:15.522067 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7f378-029f-4352-9e71-5746991a03f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.522347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7f378-029f-4352-9e71-5746991a03f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.522623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7f378-029f-4352-9e71-5746991a03f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.523357 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7f378-029f-4352-9e71-5746991a03f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:46:15.523706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7f378-029f-4352-9e71-5746991a03f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:15.525339 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7f378-029f-4352-9e71-5746991a03f9 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:15.537494 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1375/2760] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 02 03:46:15.542223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcf7ab-b3b3-4a2a-9dd0-eaec805a7607 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:15.552084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcf7ab-b3b3-4a2a-9dd0-eaec805a7607 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:15.557853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5614a-7715-4ea4-9f29-48f93e58e1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:15.558520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5614a-7715-4ea4-9f29-48f93e58e1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.558818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5614a-7715-4ea4-9f29-48f93e58e1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.559114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5614a-7715-4ea4-9f29-48f93e58e1a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.559845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5614a-7715-4ea4-9f29-48f93e58e1a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:15.561761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5614a-7715-4ea4-9f29-48f93e58e1a3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:15.575712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:15.591488 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1386/2761] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:15 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:15.596330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7f378-029f-4352-9e71-5746991a03f9 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:15.606633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7f378-029f-4352-9e71-5746991a03f9 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:15.654590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:15.670736 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ab477-27c0-4dba-92b0-04fbdaff1f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:15.671763 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ab477-27c0-4dba-92b0-04fbdaff1f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.672147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ab477-27c0-4dba-92b0-04fbdaff1f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.672541 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ab477-27c0-4dba-92b0-04fbdaff1f77 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.673765 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ab477-27c0-4dba-92b0-04fbdaff1f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:15.675876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ab477-27c0-4dba-92b0-04fbdaff1f77 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:15.703482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1376/2762] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:15 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:15.707727 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5614a-7715-4ea4-9f29-48f93e58e1a3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:15.720580 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5614a-7715-4ea4-9f29-48f93e58e1a3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:15.727546 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d3041-01cc-45ef-ac84-2ddc900d9997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:15.728273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d3041-01cc-45ef-ac84-2ddc900d9997 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.728545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d3041-01cc-45ef-ac84-2ddc900d9997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.728812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d3041-01cc-45ef-ac84-2ddc900d9997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.729564 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d3041-01cc-45ef-ac84-2ddc900d9997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:15.731205 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d3041-01cc-45ef-ac84-2ddc900d9997 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:15.743673 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1387/2763] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:15.747944 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ab477-27c0-4dba-92b0-04fbdaff1f77 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:15.761147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ab477-27c0-4dba-92b0-04fbdaff1f77 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:15.768620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6d4e8-8092-4df0-a8ab-d034ced2677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:15.769433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6d4e8-8092-4df0-a8ab-d034ced2677e admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.769718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6d4e8-8092-4df0-a8ab-d034ced2677e admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.769991 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6d4e8-8092-4df0-a8ab-d034ced2677e admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.770701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6d4e8-8092-4df0-a8ab-d034ced2677e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:15.772623 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6d4e8-8092-4df0-a8ab-d034ced2677e admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:15.784657 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1377/2764] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:15.790705 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d3041-01cc-45ef-ac84-2ddc900d9997 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:15.801472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d3041-01cc-45ef-ac84-2ddc900d9997 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:15.806318 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230c67e-4c58-424f-aa00-c42f19d61d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:15.807192 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230c67e-4c58-424f-aa00-c42f19d61d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.807512 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230c67e-4c58-424f-aa00-c42f19d61d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.807795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230c67e-4c58-424f-aa00-c42f19d61d21 admin admin] PATH_INFO: `/v3/projects/f6a38de2c8c04da5a1c75882ffbaf8e4/users/c20831efbebf484393dc3fad64e120ae/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.823092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230c67e-4c58-424f-aa00-c42f19d61d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a38de2c8c04da5a1c75882ffbaf8e4, user_id=c20831efbebf484393dc3fad64e120ae, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:15.825776 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230c67e-4c58-424f-aa00-c42f19d61d21 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:15.840741 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1388/2765] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:15 2022] PUT /identity/v3/projects/f6a38de2c8c04da5a1c75882ffbaf8e4/users/c20831efbebf484393dc3fad64e120ae/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:15.856911 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfcce6-1e16-439d-924e-9ae99e87d29a None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:15.857512 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfcce6-1e16-439d-924e-9ae99e87d29a None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:15.857848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfcce6-1e16-439d-924e-9ae99e87d29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:15.886767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-c5dfcce6-1e16-439d-924e-9ae99e87d29a None None] MFA Rules not processed for user `c20831efbebf484393dc3fad64e120ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:15.888041 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-c5dfcce6-1e16-439d-924e-9ae99e87d29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:15.956029 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1378/2766] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:15.960671 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230c67e-4c58-424f-aa00-c42f19d61d21 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:16.035635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230c67e-4c58-424f-aa00-c42f19d61d21 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:16.042105 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43002083-4ad2-4d0d-b517-c60dc08b4d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:16.042769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43002083-4ad2-4d0d-b517-c60dc08b4d13 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:16.043055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43002083-4ad2-4d0d-b517-c60dc08b4d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:16.043325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43002083-4ad2-4d0d-b517-c60dc08b4d13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:16.044042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43002083-4ad2-4d0d-b517-c60dc08b4d13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:16.045966 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43002083-4ad2-4d0d-b517-c60dc08b4d13 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:16.078659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1389/2767] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:15 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:16.083311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfcce6-1e16-439d-924e-9ae99e87d29a None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:16.097514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfcce6-1e16-439d-924e-9ae99e87d29a None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:16.172510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17aee7-226c-46d1-93e2-60a64a68d290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:16.173216 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17aee7-226c-46d1-93e2-60a64a68d290 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:16.173604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17aee7-226c-46d1-93e2-60a64a68d290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:16.174308 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17aee7-226c-46d1-93e2-60a64a68d290 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:16.175027 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17aee7-226c-46d1-93e2-60a64a68d290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:16.177063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17aee7-226c-46d1-93e2-60a64a68d290 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:16.200913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1379/2768] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:16 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:16.205797 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43002083-4ad2-4d0d-b517-c60dc08b4d13 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:16.217288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43002083-4ad2-4d0d-b517-c60dc08b4d13 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:16.222464 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083cb6d-116c-499b-a2a8-e00b1cfb61ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:16.223158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083cb6d-116c-499b-a2a8-e00b1cfb61ac admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:16.223484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083cb6d-116c-499b-a2a8-e00b1cfb61ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:16.223767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083cb6d-116c-499b-a2a8-e00b1cfb61ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:16.224509 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083cb6d-116c-499b-a2a8-e00b1cfb61ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:16.226141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3083cb6d-116c-499b-a2a8-e00b1cfb61ac admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:16.237127 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1390/2769] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:16.241451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17aee7-226c-46d1-93e2-60a64a68d290 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:16.252103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17aee7-226c-46d1-93e2-60a64a68d290 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:16.257425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2270940-6b01-49df-887a-669670068fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:16.258307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2270940-6b01-49df-887a-669670068fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:16.258589 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2270940-6b01-49df-887a-669670068fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:16.258864 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2270940-6b01-49df-887a-669670068fc3 admin admin] PATH_INFO: `/v3/projects/9fadadf4fa1e4f60b48574a9b6dfa0a3/users/f8de3f8b6250494ca1984499300752c0/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:16.274232 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2270940-6b01-49df-887a-669670068fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fadadf4fa1e4f60b48574a9b6dfa0a3, user_id=f8de3f8b6250494ca1984499300752c0, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:16.277010 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2270940-6b01-49df-887a-669670068fc3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:16.293227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1380/2770] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:16 2022] PUT /identity/v3/projects/9fadadf4fa1e4f60b48574a9b6dfa0a3/users/f8de3f8b6250494ca1984499300752c0/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:16.297451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083cb6d-116c-499b-a2a8-e00b1cfb61ac admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:16.308342 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083cb6d-116c-499b-a2a8-e00b1cfb61ac admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:16.367958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0544c0-a948-4edd-9420-c4dda7bcdc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:16.368690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0544c0-a948-4edd-9420-c4dda7bcdc64 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:16.368979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0544c0-a948-4edd-9420-c4dda7bcdc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:16.369253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0544c0-a948-4edd-9420-c4dda7bcdc64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:16.369979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0544c0-a948-4edd-9420-c4dda7bcdc64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:16.371609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0544c0-a948-4edd-9420-c4dda7bcdc64 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:16.383081 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1391/2771] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:16.388013 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2270940-6b01-49df-887a-669670068fc3 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:16.400927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2270940-6b01-49df-887a-669670068fc3 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:16.409492 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1e10c-d519-48f3-82f9-b49477a9e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:16.410759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1e10c-d519-48f3-82f9-b49477a9e2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:16.411040 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1e10c-d519-48f3-82f9-b49477a9e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:16.411388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c1e10c-d519-48f3-82f9-b49477a9e2dd admin admin] PATH_INFO: `/v3/projects/9fadadf4fa1e4f60b48574a9b6dfa0a3/users/f8de3f8b6250494ca1984499300752c0/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:16.414693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1e10c-d519-48f3-82f9-b49477a9e2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fadadf4fa1e4f60b48574a9b6dfa0a3, user_id=f8de3f8b6250494ca1984499300752c0, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:16.419356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c1e10c-d519-48f3-82f9-b49477a9e2dd admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:16.432373 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1381/2772] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:16 2022] PUT /identity/v3/projects/9fadadf4fa1e4f60b48574a9b6dfa0a3/users/f8de3f8b6250494ca1984499300752c0/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:16.500672 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a5239-5cc8-481b-89ff-f0d5fdf05b0e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:16.501140 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a5239-5cc8-481b-89ff-f0d5fdf05b0e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:16.501716 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a5239-5cc8-481b-89ff-f0d5fdf05b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:16.535663 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9e3a5239-5cc8-481b-89ff-f0d5fdf05b0e None None] MFA Rules not processed for user `f8de3f8b6250494ca1984499300752c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:16.536892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e3a5239-5cc8-481b-89ff-f0d5fdf05b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:16.555538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:16.611123 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1392/2773] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:16 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:16.624776 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1e10c-d519-48f3-82f9-b49477a9e2dd admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:16.628021 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-65c1e10c-d519-48f3-82f9-b49477a9e2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:16.649018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:16.697544 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c1e10c-d519-48f3-82f9-b49477a9e2dd admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:16.701706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224888a0-6a13-496c-babe-99d1a6239635 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:16.702412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224888a0-6a13-496c-babe-99d1a6239635 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:16.702700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224888a0-6a13-496c-babe-99d1a6239635 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:16.702978 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224888a0-6a13-496c-babe-99d1a6239635 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:16.760898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224888a0-6a13-496c-babe-99d1a6239635 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:16.762264 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224888a0-6a13-496c-babe-99d1a6239635 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:16.771934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1382/2774] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:21.058253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a5239-5cc8-481b-89ff-f0d5fdf05b0e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:21.059794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e3a5239-5cc8-481b-89ff-f0d5fdf05b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:21.073394 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a5239-5cc8-481b-89ff-f0d5fdf05b0e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:21.077567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d411fe-a6de-411d-bad1-7066b925b249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:21.078345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d411fe-a6de-411d-bad1-7066b925b249 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:21.078634 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d411fe-a6de-411d-bad1-7066b925b249 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:21.078927 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d411fe-a6de-411d-bad1-7066b925b249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:21.088893 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d411fe-a6de-411d-bad1-7066b925b249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:21.090340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d411fe-a6de-411d-bad1-7066b925b249 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:21.101777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1393/2775] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:23.382920 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f461-f547-4014-8a9b-276e285bcef7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:23.383466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f461-f547-4014-8a9b-276e285bcef7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:23.383466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614f461-f547-4014-8a9b-276e285bcef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:23.401799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7614f461-f547-4014-8a9b-276e285bcef7 None None] MFA Rules not processed for user `d42b42214d0d46bbb9fe90aed7f5e382`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:23.403024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7614f461-f547-4014-8a9b-276e285bcef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:23.461800 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1383/2776] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:23.788469 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d411fe-a6de-411d-bad1-7066b925b249 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:23.790207 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b4d411fe-a6de-411d-bad1-7066b925b249 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:23.854988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d411fe-a6de-411d-bad1-7066b925b249 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:23.858423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda515c-4c64-4e64-8c72-2cc548989aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:23.859085 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda515c-4c64-4e64-8c72-2cc548989aec service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:23.859391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda515c-4c64-4e64-8c72-2cc548989aec service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:23.859668 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda515c-4c64-4e64-8c72-2cc548989aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:23.861514 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-acda515c-4c64-4e64-8c72-2cc548989aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:23.872323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda515c-4c64-4e64-8c72-2cc548989aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:23.873768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda515c-4c64-4e64-8c72-2cc548989aec service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:23.882162 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1394/2777] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:26.075376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f461-f547-4014-8a9b-276e285bcef7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:26.182325 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=100 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:26.229531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614f461-f547-4014-8a9b-276e285bcef7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:26.236880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:26.238289 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:26.238752 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:26.239149 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] PATH_INFO: `/v3/users/c20831efbebf484393dc3fad64e120ae` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:26.241148 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c20831efbebf484393dc3fad64e120ae)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:26.244410 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:26.306524 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c20831efbebf484393dc3fad64e120ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:26.307177 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c20831efbebf484393dc3fad64e120ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:26.316058 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c20831efbebf484393dc3fad64e120ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:26.317081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c20831efbebf484393dc3fad64e120ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:26.331429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c20831efbebf484393dc3fad64e120ae'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:26.345701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1384/2778] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:26 2022] DELETE /identity/v3/users/c20831efbebf484393dc3fad64e120ae => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:26.350297 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02959f5-f49c-4c13-8c8e-1ebd3b6520a5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:26.351161 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02959f5-f49c-4c13-8c8e-1ebd3b6520a5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:26.351782 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02959f5-f49c-4c13-8c8e-1ebd3b6520a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:26.392572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e02959f5-f49c-4c13-8c8e-1ebd3b6520a5 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:26.393778 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e02959f5-f49c-4c13-8c8e-1ebd3b6520a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:26.453544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1395/2779] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:26.463913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:26.465408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:26.524236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd478d-6071-448f-bb82-12199765553a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:26.528213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7072f-6829-4e74-a626-0aadf9de680e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:26.529172 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7072f-6829-4e74-a626-0aadf9de680e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:26.529508 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7072f-6829-4e74-a626-0aadf9de680e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:26.529814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7072f-6829-4e74-a626-0aadf9de680e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:26.544926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7072f-6829-4e74-a626-0aadf9de680e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:26.546627 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7072f-6829-4e74-a626-0aadf9de680e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:26.558243 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1385/2780] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:27.011899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02959f5-f49c-4c13-8c8e-1ebd3b6520a5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:27.013565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e02959f5-f49c-4c13-8c8e-1ebd3b6520a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:27.029189 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02959f5-f49c-4c13-8c8e-1ebd3b6520a5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:27.034141 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:27.035248 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:27.035559 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:27.035837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] PATH_INFO: `/v3/projects/f6a38de2c8c04da5a1c75882ffbaf8e4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:27.037333 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a38de2c8c04da5a1c75882ffbaf8e4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:27.039279 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:27.070391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:27.100515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a38de2c8c04da5a1c75882ffbaf8e4', 'request_id': 'req-8e7a142f-d784-4e38-99b5-344393a4a90c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.111545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a38de2c8c04da5a1c75882ffbaf8e4', 'request_id': 'req-8e7a142f-d784-4e38-99b5-344393a4a90c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.118866 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a38de2c8c04da5a1c75882ffbaf8e4', 'request_id': 'req-8e7a142f-d784-4e38-99b5-344393a4a90c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.137501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] The token cache is being invalidate because project f6a38de2c8c04da5a1c75882ffbaf8e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:27.137831 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.138694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.140755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1396/2781] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:27 2022] DELETE /identity/v3/projects/f6a38de2c8c04da5a1c75882ffbaf8e4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:27.144845 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7072f-6829-4e74-a626-0aadf9de680e service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:27.155015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7072f-6829-4e74-a626-0aadf9de680e service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:27.203317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:27.214814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:27.215788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:27.216256 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:27.216682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] PATH_INFO: `/v3/users/f8de3f8b6250494ca1984499300752c0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:27.218049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8de3f8b6250494ca1984499300752c0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:27.220089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:27.271685 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8de3f8b6250494ca1984499300752c0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.272399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8de3f8b6250494ca1984499300752c0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.282923 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8de3f8b6250494ca1984499300752c0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.283584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8de3f8b6250494ca1984499300752c0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.297167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-012373b8-09e3-40e6-913f-b22f40ebfcd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8de3f8b6250494ca1984499300752c0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.310848 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1386/2782] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:27 2022] DELETE /identity/v3/users/f8de3f8b6250494ca1984499300752c0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:27.592688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:27.594329 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:27.610908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a142f-d784-4e38-99b5-344393a4a90c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:27.677216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:27.678091 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:27.678370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:27.678642 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] PATH_INFO: `/v3/projects/9fadadf4fa1e4f60b48574a9b6dfa0a3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:27.679867 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fadadf4fa1e4f60b48574a9b6dfa0a3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:27.681794 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:27.730558 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fadadf4fa1e4f60b48574a9b6dfa0a3', 'request_id': 'req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.735989 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fadadf4fa1e4f60b48574a9b6dfa0a3', 'request_id': 'req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.742296 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fadadf4fa1e4f60b48574a9b6dfa0a3', 'request_id': 'req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.756491 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] The token cache is being invalidate because project 9fadadf4fa1e4f60b48574a9b6dfa0a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:27.756774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.757397 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-349ea2ee-9bde-4223-a0dd-b23a5f0376bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:27.758969 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1397/2783] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:27 2022] DELETE /identity/v3/projects/9fadadf4fa1e4f60b48574a9b6dfa0a3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:27.765265 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc18f10-4a2d-42ea-b2a8-a5f6d532946d None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:27.765591 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc18f10-4a2d-42ea-b2a8-a5f6d532946d None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:27.765870 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc18f10-4a2d-42ea-b2a8-a5f6d532946d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:27.814490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9cc18f10-4a2d-42ea-b2a8-a5f6d532946d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:27.815820 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9cc18f10-4a2d-42ea-b2a8-a5f6d532946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:27.880356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1387/2784] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:27.888652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076f45b-332e-4e54-9037-9546071de77e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:27.888940 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076f45b-332e-4e54-9037-9546071de77e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:27.889223 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076f45b-332e-4e54-9037-9546071de77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:27.908052 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d076f45b-332e-4e54-9037-9546071de77e None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:27.909265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d076f45b-332e-4e54-9037-9546071de77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:27.919092 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1398/2785] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:27.923675 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc18f10-4a2d-42ea-b2a8-a5f6d532946d None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:27.934576 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc18f10-4a2d-42ea-b2a8-a5f6d532946d None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:27.940263 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d18768-942f-4b24-bf3a-71b891d12fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:27.940886 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18768-942f-4b24-bf3a-71b891d12fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:27.941173 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18768-942f-4b24-bf3a-71b891d12fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:27.941452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18768-942f-4b24-bf3a-71b891d12fd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:27.942193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d18768-942f-4b24-bf3a-71b891d12fd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:46:27.942537 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d18768-942f-4b24-bf3a-71b891d12fd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:27.944384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d18768-942f-4b24-bf3a-71b891d12fd6 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:27.958470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1388/2786] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:46:27.963476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076f45b-332e-4e54-9037-9546071de77e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:27.973075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076f45b-332e-4e54-9037-9546071de77e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:27.977994 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a9d96-bd27-4421-8de5-7554a47b6353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:27.978648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a9d96-bd27-4421-8de5-7554a47b6353 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:27.978929 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a9d96-bd27-4421-8de5-7554a47b6353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:27.979229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a9d96-bd27-4421-8de5-7554a47b6353 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:27.980462 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a9d96-bd27-4421-8de5-7554a47b6353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:27.982384 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a9d96-bd27-4421-8de5-7554a47b6353 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.010265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1399/2787] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:27 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:28.013862 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d18768-942f-4b24-bf3a-71b891d12fd6 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.024969 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d18768-942f-4b24-bf3a-71b891d12fd6 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.057844 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:28.086407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebabc9-9c63-48c0-a662-beaba1d7f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.087081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebabc9-9c63-48c0-a662-beaba1d7f5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.087393 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebabc9-9c63-48c0-a662-beaba1d7f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.087671 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebabc9-9c63-48c0-a662-beaba1d7f5a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.088371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebabc9-9c63-48c0-a662-beaba1d7f5a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.090298 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebabc9-9c63-48c0-a662-beaba1d7f5a7 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.113349 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1389/2788] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:28 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:46:28.117963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a9d96-bd27-4421-8de5-7554a47b6353 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.129348 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a9d96-bd27-4421-8de5-7554a47b6353 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.135015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bea621-a0ca-449f-9c67-3ee76742dd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.135798 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bea621-a0ca-449f-9c67-3ee76742dd7a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.136078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bea621-a0ca-449f-9c67-3ee76742dd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.136346 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bea621-a0ca-449f-9c67-3ee76742dd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.137089 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bea621-a0ca-449f-9c67-3ee76742dd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.138719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bea621-a0ca-449f-9c67-3ee76742dd7a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.141115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:28.151696 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1400/2789] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:28.156136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebabc9-9c63-48c0-a662-beaba1d7f5a7 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.167503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebabc9-9c63-48c0-a662-beaba1d7f5a7 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.173465 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31dc46-6c56-4ef7-b801-590a20e6c33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.174115 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31dc46-6c56-4ef7-b801-590a20e6c33f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.174407 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31dc46-6c56-4ef7-b801-590a20e6c33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.174692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31dc46-6c56-4ef7-b801-590a20e6c33f admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.175452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31dc46-6c56-4ef7-b801-590a20e6c33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.177144 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31dc46-6c56-4ef7-b801-590a20e6c33f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.187716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1390/2790] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:28.192006 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bea621-a0ca-449f-9c67-3ee76742dd7a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.202731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bea621-a0ca-449f-9c67-3ee76742dd7a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.208726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55525874-57a1-4813-bb62-6b795f4e3bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.209608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525874-57a1-4813-bb62-6b795f4e3bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.209887 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525874-57a1-4813-bb62-6b795f4e3bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.210158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55525874-57a1-4813-bb62-6b795f4e3bd6 admin admin] PATH_INFO: `/v3/projects/48fa35f0c4d445169a54c5e428c3a829/users/8384e7eaa4394a48a38ce4f3e971fee6/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.225758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55525874-57a1-4813-bb62-6b795f4e3bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48fa35f0c4d445169a54c5e428c3a829, user_id=8384e7eaa4394a48a38ce4f3e971fee6, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.228484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55525874-57a1-4813-bb62-6b795f4e3bd6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.245954 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1401/2791] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:28 2022] PUT /identity/v3/projects/48fa35f0c4d445169a54c5e428c3a829/users/8384e7eaa4394a48a38ce4f3e971fee6/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:28.258784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbef49d-aa4d-4490-9b94-47cdcec66ac2 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.259080 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbef49d-aa4d-4490-9b94-47cdcec66ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.259414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbef49d-aa4d-4490-9b94-47cdcec66ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.287625 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-edbef49d-aa4d-4490-9b94-47cdcec66ac2 None None] MFA Rules not processed for user `8384e7eaa4394a48a38ce4f3e971fee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:28.288843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-edbef49d-aa4d-4490-9b94-47cdcec66ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:28.357731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1391/2792] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:28.362608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55525874-57a1-4813-bb62-6b795f4e3bd6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.425315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55525874-57a1-4813-bb62-6b795f4e3bd6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.430575 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f834ba-6bdf-43d4-a1b2-6b118fa513f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.431226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f834ba-6bdf-43d4-a1b2-6b118fa513f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.431538 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f834ba-6bdf-43d4-a1b2-6b118fa513f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.431813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f834ba-6bdf-43d4-a1b2-6b118fa513f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.432499 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f834ba-6bdf-43d4-a1b2-6b118fa513f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.434417 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f834ba-6bdf-43d4-a1b2-6b118fa513f5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.462255 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1402/2793] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:28 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:28.466441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbef49d-aa4d-4490-9b94-47cdcec66ac2 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.477594 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbef49d-aa4d-4490-9b94-47cdcec66ac2 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.539007 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb14380-5a68-416f-924e-62ff154ccd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.539730 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb14380-5a68-416f-924e-62ff154ccd7c admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.540011 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb14380-5a68-416f-924e-62ff154ccd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.540293 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb14380-5a68-416f-924e-62ff154ccd7c admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.540998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb14380-5a68-416f-924e-62ff154ccd7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.543269 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb14380-5a68-416f-924e-62ff154ccd7c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.565326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1392/2794] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:28 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:28.569517 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f834ba-6bdf-43d4-a1b2-6b118fa513f5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.580490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f834ba-6bdf-43d4-a1b2-6b118fa513f5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.586784 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad189a-f581-4c81-8ca4-51fd36b7a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.587513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad189a-f581-4c81-8ca4-51fd36b7a154 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.587799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad189a-f581-4c81-8ca4-51fd36b7a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.588078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad189a-f581-4c81-8ca4-51fd36b7a154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.588809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad189a-f581-4c81-8ca4-51fd36b7a154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.590455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad189a-f581-4c81-8ca4-51fd36b7a154 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.602311 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1403/2795] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:28.606462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb14380-5a68-416f-924e-62ff154ccd7c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.617358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb14380-5a68-416f-924e-62ff154ccd7c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.622455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e068d-254e-4753-928c-51a5eb8db970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.623450 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e068d-254e-4753-928c-51a5eb8db970 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.623742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e068d-254e-4753-928c-51a5eb8db970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.624018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e068d-254e-4753-928c-51a5eb8db970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.624744 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e068d-254e-4753-928c-51a5eb8db970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.626556 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e068d-254e-4753-928c-51a5eb8db970 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.636885 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1393/2796] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:28.641303 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad189a-f581-4c81-8ca4-51fd36b7a154 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.652286 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad189a-f581-4c81-8ca4-51fd36b7a154 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.657644 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd95f9f-f8c5-4ae6-97be-1dd42478146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.658545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd95f9f-f8c5-4ae6-97be-1dd42478146e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.658853 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd95f9f-f8c5-4ae6-97be-1dd42478146e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.659130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd95f9f-f8c5-4ae6-97be-1dd42478146e admin admin] PATH_INFO: `/v3/projects/41e2bd5c4e3a472e85544d643923a445/users/91228d9262aa4923b9f45ebab1426bfe/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.675281 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd95f9f-f8c5-4ae6-97be-1dd42478146e admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e2bd5c4e3a472e85544d643923a445, user_id=91228d9262aa4923b9f45ebab1426bfe, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.678325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd95f9f-f8c5-4ae6-97be-1dd42478146e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.692975 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1404/2797] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:28 2022] PUT /identity/v3/projects/41e2bd5c4e3a472e85544d643923a445/users/91228d9262aa4923b9f45ebab1426bfe/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:28.703498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b673522-4231-4eb2-b197-e96377a13a5b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.703960 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b673522-4231-4eb2-b197-e96377a13a5b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.704312 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b673522-4231-4eb2-b197-e96377a13a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.732136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9b673522-4231-4eb2-b197-e96377a13a5b None None] MFA Rules not processed for user `91228d9262aa4923b9f45ebab1426bfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:28.733358 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9b673522-4231-4eb2-b197-e96377a13a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:28.744529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:28.806917 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1394/2798] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:28.811474 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd95f9f-f8c5-4ae6-97be-1dd42478146e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.876739 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd95f9f-f8c5-4ae6-97be-1dd42478146e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.882515 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7633b-5039-4b30-af36-15f1da90bfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.883183 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7633b-5039-4b30-af36-15f1da90bfd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.883495 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7633b-5039-4b30-af36-15f1da90bfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.883773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7633b-5039-4b30-af36-15f1da90bfd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.884476 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7633b-5039-4b30-af36-15f1da90bfd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.886390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7633b-5039-4b30-af36-15f1da90bfd8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:28.916935 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1405/2799] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:46:28 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:28.918303 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b673522-4231-4eb2-b197-e96377a13a5b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:28.929422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b673522-4231-4eb2-b197-e96377a13a5b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:28.986981 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c011ef9-b4eb-41e3-859b-91fdc9674c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:28.987928 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c011ef9-b4eb-41e3-859b-91fdc9674c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:28.988218 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c011ef9-b4eb-41e3-859b-91fdc9674c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:28.988491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c011ef9-b4eb-41e3-859b-91fdc9674c9a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:28.989189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c011ef9-b4eb-41e3-859b-91fdc9674c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:28.991169 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c011ef9-b4eb-41e3-859b-91fdc9674c9a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:29.016722 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1395/2800] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:46:28 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:46:29.018321 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7633b-5039-4b30-af36-15f1da90bfd8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:29.030659 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7633b-5039-4b30-af36-15f1da90bfd8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:29.035872 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84517a3-52db-4034-87ae-9c4315cdef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:29.036621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84517a3-52db-4034-87ae-9c4315cdef53 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:29.036908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84517a3-52db-4034-87ae-9c4315cdef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:29.037178 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84517a3-52db-4034-87ae-9c4315cdef53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:29.037908 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84517a3-52db-4034-87ae-9c4315cdef53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:29.039587 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84517a3-52db-4034-87ae-9c4315cdef53 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:29.043240 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:29.058107 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1406/2801] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 02 03:46:29.059932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c011ef9-b4eb-41e3-859b-91fdc9674c9a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:29.078296 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c011ef9-b4eb-41e3-859b-91fdc9674c9a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:29.083566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95caa2-c9a0-4ef5-b93f-7ae0da571c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:29.084430 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95caa2-c9a0-4ef5-b93f-7ae0da571c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:29.084718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95caa2-c9a0-4ef5-b93f-7ae0da571c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:29.085340 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95caa2-c9a0-4ef5-b93f-7ae0da571c3c admin admin] PATH_INFO: `/v3/projects/af3d452e5085499aa20cdb21292aac19/users/ae63cb6f589d4f69832477653c20cfb4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:29.100919 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95caa2-c9a0-4ef5-b93f-7ae0da571c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3d452e5085499aa20cdb21292aac19, user_id=ae63cb6f589d4f69832477653c20cfb4, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:29.103774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95caa2-c9a0-4ef5-b93f-7ae0da571c3c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:29.121047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1396/2802] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:29 2022] PUT /identity/v3/projects/af3d452e5085499aa20cdb21292aac19/users/ae63cb6f589d4f69832477653c20cfb4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:29.125370 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84517a3-52db-4034-87ae-9c4315cdef53 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:29.141327 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84517a3-52db-4034-87ae-9c4315cdef53 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:29.198158 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e2d89-226f-4bac-9d51-758858643c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:29.198838 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e2d89-226f-4bac-9d51-758858643c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:29.199172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e2d89-226f-4bac-9d51-758858643c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:29.199475 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e2d89-226f-4bac-9d51-758858643c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:29.200181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e2d89-226f-4bac-9d51-758858643c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:29.201811 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e2d89-226f-4bac-9d51-758858643c9d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:29.212615 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1407/2803] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:46:29.216898 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95caa2-c9a0-4ef5-b93f-7ae0da571c3c admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:29.228145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95caa2-c9a0-4ef5-b93f-7ae0da571c3c admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:29.233201 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0bf97-0fd5-4526-8c72-d5f0c5df847b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:29.234110 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0bf97-0fd5-4526-8c72-d5f0c5df847b admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:29.234396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0bf97-0fd5-4526-8c72-d5f0c5df847b admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:29.234682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0bf97-0fd5-4526-8c72-d5f0c5df847b admin admin] PATH_INFO: `/v3/projects/af3d452e5085499aa20cdb21292aac19/users/ae63cb6f589d4f69832477653c20cfb4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:29.236949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0bf97-0fd5-4526-8c72-d5f0c5df847b admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3d452e5085499aa20cdb21292aac19, user_id=ae63cb6f589d4f69832477653c20cfb4, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:29.239806 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0bf97-0fd5-4526-8c72-d5f0c5df847b admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:29.250992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1397/2804] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:46:29 2022] PUT /identity/v3/projects/af3d452e5085499aa20cdb21292aac19/users/ae63cb6f589d4f69832477653c20cfb4/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:29.261497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511fed2-43ea-4c68-a064-2b7718a798c6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:29.261789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511fed2-43ea-4c68-a064-2b7718a798c6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:29.262077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511fed2-43ea-4c68-a064-2b7718a798c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:29.290442 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e511fed2-43ea-4c68-a064-2b7718a798c6 None None] MFA Rules not processed for user `ae63cb6f589d4f69832477653c20cfb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:29.292149 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e511fed2-43ea-4c68-a064-2b7718a798c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:29.363062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1408/2805] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:29.375066 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0bf97-0fd5-4526-8c72-d5f0c5df847b admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:29.376927 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4dc0bf97-0fd5-4526-8c72-d5f0c5df847b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:29.457713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:29.462015 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0bf97-0fd5-4526-8c72-d5f0c5df847b admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:29.466867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba346db1-fd2d-4be0-aabb-7649965e1da1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:29.467584 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba346db1-fd2d-4be0-aabb-7649965e1da1 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:29.467861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba346db1-fd2d-4be0-aabb-7649965e1da1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:29.468130 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba346db1-fd2d-4be0-aabb-7649965e1da1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:29.526400 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba346db1-fd2d-4be0-aabb-7649965e1da1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:29.527921 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba346db1-fd2d-4be0-aabb-7649965e1da1 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:29.538432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1398/2806] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:31.084543 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511fed2-43ea-4c68-a064-2b7718a798c6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:31.086143 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e511fed2-43ea-4c68-a064-2b7718a798c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:31.099088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511fed2-43ea-4c68-a064-2b7718a798c6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:31.103721 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b9b5a-2f7a-46e0-963a-64d35d14d755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:31.104373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b9b5a-2f7a-46e0-963a-64d35d14d755 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:31.104653 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b9b5a-2f7a-46e0-963a-64d35d14d755 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:31.104937 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b9b5a-2f7a-46e0-963a-64d35d14d755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:31.165972 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b9b5a-2f7a-46e0-963a-64d35d14d755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:31.167747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b9b5a-2f7a-46e0-963a-64d35d14d755 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:31.176918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1409/2807] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:31.728283 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba346db1-fd2d-4be0-aabb-7649965e1da1 service cinder] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:31.737814 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba346db1-fd2d-4be0-aabb-7649965e1da1 service cinder] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:31.743477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e7418-0637-437e-b441-883107d4ab37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:31.744145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e7418-0637-437e-b441-883107d4ab37 service cinder] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:31.744433 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e7418-0637-437e-b441-883107d4ab37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:31.744707 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e7418-0637-437e-b441-883107d4ab37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:31.755778 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e7418-0637-437e-b441-883107d4ab37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:31.757388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e7418-0637-437e-b441-883107d4ab37 service cinder] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:31.767432 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1399/2808] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:35.722987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b9b5a-2f7a-46e0-963a-64d35d14d755 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:35.748436 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:35.797554 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b9b5a-2f7a-46e0-963a-64d35d14d755 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:35.802265 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:35.803168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:35.803496 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:35.803775 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] PATH_INFO: `/v3/users/8384e7eaa4394a48a38ce4f3e971fee6` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:35.804979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8384e7eaa4394a48a38ce4f3e971fee6)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:35.807501 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:35.856645 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8384e7eaa4394a48a38ce4f3e971fee6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:35.857382 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8384e7eaa4394a48a38ce4f3e971fee6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:35.867828 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8384e7eaa4394a48a38ce4f3e971fee6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:35.880876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8384e7eaa4394a48a38ce4f3e971fee6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:35.881537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8384e7eaa4394a48a38ce4f3e971fee6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:35.894037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1410/2809] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:35 2022] DELETE /identity/v3/users/8384e7eaa4394a48a38ce4f3e971fee6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:35.898949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fd6b9-f8d2-4541-b780-b3c61b6709e4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:35.899534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fd6b9-f8d2-4541-b780-b3c61b6709e4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:35.899534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fd6b9-f8d2-4541-b780-b3c61b6709e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:35.917723 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5a0fd6b9-f8d2-4541-b780-b3c61b6709e4 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:46:35.918926 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5a0fd6b9-f8d2-4541-b780-b3c61b6709e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:35.983307 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1400/2810] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:46:35.993441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:35.995011 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:36.098145 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4a1e4-f532-49df-b150-d18be114efaa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:36.101978 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587b016-db76-42f7-8a6c-9b2c1849b929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:36.102677 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587b016-db76-42f7-8a6c-9b2c1849b929 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:36.103068 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587b016-db76-42f7-8a6c-9b2c1849b929 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:36.103378 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587b016-db76-42f7-8a6c-9b2c1849b929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:36.118579 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587b016-db76-42f7-8a6c-9b2c1849b929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:36.120156 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587b016-db76-42f7-8a6c-9b2c1849b929 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:36.133201 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1411/2811] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:36.505122 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fd6b9-f8d2-4541-b780-b3c61b6709e4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:36.506590 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5a0fd6b9-f8d2-4541-b780-b3c61b6709e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:36.519762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fd6b9-f8d2-4541-b780-b3c61b6709e4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:36.525460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:36.526346 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:36.526648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:36.526902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] PATH_INFO: `/v3/projects/48fa35f0c4d445169a54c5e428c3a829` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:36.528239 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48fa35f0c4d445169a54c5e428c3a829)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:36.530183 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:36.581236 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48fa35f0c4d445169a54c5e428c3a829', 'request_id': 'req-adb2c811-0908-4b6e-a713-fa7cf06fde68'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.590317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48fa35f0c4d445169a54c5e428c3a829', 'request_id': 'req-adb2c811-0908-4b6e-a713-fa7cf06fde68'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.596633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48fa35f0c4d445169a54c5e428c3a829', 'request_id': 'req-adb2c811-0908-4b6e-a713-fa7cf06fde68'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.607414 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] The token cache is being invalidate because project 48fa35f0c4d445169a54c5e428c3a829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:36.607701 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.608386 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.609954 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1401/2812] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:36 2022] DELETE /identity/v3/projects/48fa35f0c4d445169a54c5e428c3a829 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:46:36.614037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587b016-db76-42f7-8a6c-9b2c1849b929 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:36.624594 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587b016-db76-42f7-8a6c-9b2c1849b929 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:36.641217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:36.686239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:36.687641 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:36.688072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:36.688583 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] PATH_INFO: `/v3/users/91228d9262aa4923b9f45ebab1426bfe` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:36.690621 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=91228d9262aa4923b9f45ebab1426bfe)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:36.693761 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:36.750114 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91228d9262aa4923b9f45ebab1426bfe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.750783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91228d9262aa4923b9f45ebab1426bfe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.760106 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91228d9262aa4923b9f45ebab1426bfe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.771936 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91228d9262aa4923b9f45ebab1426bfe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.772569 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91228d9262aa4923b9f45ebab1426bfe'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:36.785483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1412/2813] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:36 2022] DELETE /identity/v3/users/91228d9262aa4923b9f45ebab1426bfe => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:37.074532 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:37.076388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:37.090154 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2c811-0908-4b6e-a713-fa7cf06fde68 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:37.100471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:37.153771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:37.154705 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:37.154988 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:37.155268 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] PATH_INFO: `/v3/projects/41e2bd5c4e3a472e85544d643923a445` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:37.156518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e2bd5c4e3a472e85544d643923a445)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:37.158519 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:37.208055 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e2bd5c4e3a472e85544d643923a445', 'request_id': 'req-fb3d3164-2987-4468-bd8d-eddd36a412b0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.217053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e2bd5c4e3a472e85544d643923a445', 'request_id': 'req-fb3d3164-2987-4468-bd8d-eddd36a412b0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.223670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e2bd5c4e3a472e85544d643923a445', 'request_id': 'req-fb3d3164-2987-4468-bd8d-eddd36a412b0'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.233859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] The token cache is being invalidate because project 41e2bd5c4e3a472e85544d643923a445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:37.234146 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.234775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.236425 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1402/2814] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:37 2022] DELETE /identity/v3/projects/41e2bd5c4e3a472e85544d643923a445 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:37.240813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:37.242352 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:37.256121 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa44533-0902-45ae-92ef-b04e137f0e1e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:37.327239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:37.328241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:37.328519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:37.328795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] PATH_INFO: `/v3/users/ae63cb6f589d4f69832477653c20cfb4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:37.329993 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae63cb6f589d4f69832477653c20cfb4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:37.332040 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:37.375177 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:37.381550 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae63cb6f589d4f69832477653c20cfb4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.382384 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae63cb6f589d4f69832477653c20cfb4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.391907 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae63cb6f589d4f69832477653c20cfb4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.406503 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae63cb6f589d4f69832477653c20cfb4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.407435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae63cb6f589d4f69832477653c20cfb4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.428726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1413/2815] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:46:37 2022] DELETE /identity/v3/users/ae63cb6f589d4f69832477653c20cfb4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:37.754818 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:37.756697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:37.772716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d3164-2987-4468-bd8d-eddd36a412b0 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:37.851513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:37.853016 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:37.853622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:37.854142 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] PATH_INFO: `/v3/projects/af3d452e5085499aa20cdb21292aac19` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:37.855750 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3d452e5085499aa20cdb21292aac19)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:37.857945 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:37.905403 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:37.908464 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3d452e5085499aa20cdb21292aac19', 'request_id': 'req-d0b90dbf-6a39-400f-9995-ec2d9f313077'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.918102 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3d452e5085499aa20cdb21292aac19', 'request_id': 'req-d0b90dbf-6a39-400f-9995-ec2d9f313077'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.925718 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3d452e5085499aa20cdb21292aac19', 'request_id': 'req-d0b90dbf-6a39-400f-9995-ec2d9f313077'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.936642 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] The token cache is being invalidate because project af3d452e5085499aa20cdb21292aac19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:46:37.936934 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.937615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:46:37.939175 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1403/2816] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:46:37 2022] DELETE /identity/v3/projects/af3d452e5085499aa20cdb21292aac19 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:46:41.911197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:46:41.912755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:41.991498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75fb46-d16b-44fa-95e7-f1c165a78a87 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:46:42.059468 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:46:42.060134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:46:42.060418 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:46:42.060695 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:46:42.062812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:46:42.076030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:46:42.077651 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:46:42.088846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1414/2817] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:46:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:46:47.933771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:46:52.084056 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=76,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:48:25.254644 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:48:25.255973 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:48:25.357396 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b90dbf-6a39-400f-9995-ec2d9f313077 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:48:25.362682 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d297ad-cb2a-4ce5-805c-701cb93dd538 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:48:25.363555 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d297ad-cb2a-4ce5-805c-701cb93dd538 service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:48:25.363998 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d297ad-cb2a-4ce5-805c-701cb93dd538 service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:48:25.364302 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d297ad-cb2a-4ce5-805c-701cb93dd538 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:48:25.366189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-34d297ad-cb2a-4ce5-805c-701cb93dd538 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:48:25.378687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d297ad-cb2a-4ce5-805c-701cb93dd538 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:48:25.380598 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d297ad-cb2a-4ce5-805c-701cb93dd538 service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:48:25.394731 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1404/2818] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:48:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:48:31.002139 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:48:31.003812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:48:31.096742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e8cc9-d39d-41ce-81cf-72674bcbf474 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:48:31.101646 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e02b8a64165541889ca929b8038412bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:48:31.102299 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:48:31.102576 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:48:31.102846 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:48:31.105195 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:48:31.172531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:48:31.173991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:48:31.183628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1415/2819] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:48:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:48:35.388139 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=41 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:48:41.178988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=57 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:48:42.267286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439e6ab-f84d-4a63-ac2e-d9f31dd306a7 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:48:42.267876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439e6ab-f84d-4a63-ac2e-d9f31dd306a7 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:48:42.267876 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439e6ab-f84d-4a63-ac2e-d9f31dd306a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:48:42.287333 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-e439e6ab-f84d-4a63-ac2e-d9f31dd306a7 None None] MFA Rules not processed for user `d42b42214d0d46bbb9fe90aed7f5e382`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:48:42.288580 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e439e6ab-f84d-4a63-ac2e-d9f31dd306a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:48:42.296887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1405/2820] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:48:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:48:52.281966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:49:03.202635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:49:03.205266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:49:03.282473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c41ac91-d1b1-4d27-8b1c-7c8421ba682d service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:49:03.287129 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf0674-4518-413e-884d-a88e9e3be746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:49:03.288003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf0674-4518-413e-884d-a88e9e3be746 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:49:03.288309 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf0674-4518-413e-884d-a88e9e3be746 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:49:03.288556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf0674-4518-413e-884d-a88e9e3be746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:49:03.290567 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7cbf0674-4518-413e-884d-a88e9e3be746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:49:03.370115 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf0674-4518-413e-884d-a88e9e3be746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:49:03.371767 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf0674-4518-413e-884d-a88e9e3be746 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:49:03.386197 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1416/2821] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:49:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:49:13.379498 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=50 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:50:09.067662 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439e6ab-f84d-4a63-ac2e-d9f31dd306a7 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:50:09.069156 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-e439e6ab-f84d-4a63-ac2e-d9f31dd306a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:50:09.082624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439e6ab-f84d-4a63-ac2e-d9f31dd306a7 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:50:09.086787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1350c-f04e-4dab-846e-db38f4bb6fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:50:09.087470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1350c-f04e-4dab-846e-db38f4bb6fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:50:09.087757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1350c-f04e-4dab-846e-db38f4bb6fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:50:09.088035 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1350c-f04e-4dab-846e-db38f4bb6fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:50:09.089798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0bd1350c-f04e-4dab-846e-db38f4bb6fb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:50:09.126261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1350c-f04e-4dab-846e-db38f4bb6fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:50:09.127861 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1350c-f04e-4dab-846e-db38f4bb6fb0 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:50:09.148617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1406/2822] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:50:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:50:11.527926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf0674-4518-413e-884d-a88e9e3be746 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:50:11.536925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf0674-4518-413e-884d-a88e9e3be746 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:50:11.540743 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b9ca2-609e-45c7-b586-56381b8c7939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:50:11.541423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b9ca2-609e-45c7-b586-56381b8c7939 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:50:11.542034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b9ca2-609e-45c7-b586-56381b8c7939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:50:11.542461 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b9ca2-609e-45c7-b586-56381b8c7939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:50:11.544344 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2c1b9ca2-609e-45c7-b586-56381b8c7939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:50:11.558544 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b9ca2-609e-45c7-b586-56381b8c7939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:50:11.560036 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b9ca2-609e-45c7-b586-56381b8c7939 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:50:11.574010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1417/2823] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:50:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:50:15.676893 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedace5a-6df4-4b2c-9501-2c881472e5ab None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:50:15.677441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedace5a-6df4-4b2c-9501-2c881472e5ab None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:50:15.677441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedace5a-6df4-4b2c-9501-2c881472e5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:50:15.695152 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-bedace5a-6df4-4b2c-9501-2c881472e5ab None None] MFA Rules not processed for user `e80e09005b7e4cdc8f42be866dbd91bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:50:15.696401 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-bedace5a-6df4-4b2c-9501-2c881472e5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:50:15.755039 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1407/2824] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:50:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:50:16.080852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b9ca2-609e-45c7-b586-56381b8c7939 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:50:16.090432 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b9ca2-609e-45c7-b586-56381b8c7939 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:50:16.095225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:50:16.095905 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:50:16.096180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:50:16.096452 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:50:16.098643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:50:16.109209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:50:16.110698 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:50:16.119988 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1418/2825] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:50:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:50:25.747858 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=43 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:50:26.115757 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=12 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:50:26.671384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedace5a-6df4-4b2c-9501-2c881472e5ab None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:50:26.681528 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedace5a-6df4-4b2c-9501-2c881472e5ab None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:50:26.685648 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad322-e3ef-46c6-b8fa-bd668fcf170e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:50:26.686292 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad322-e3ef-46c6-b8fa-bd668fcf170e service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:50:26.686570 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad322-e3ef-46c6-b8fa-bd668fcf170e service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:50:26.686843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ad322-e3ef-46c6-b8fa-bd668fcf170e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:50:26.688728 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-578ad322-e3ef-46c6-b8fa-bd668fcf170e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:50:26.699258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ad322-e3ef-46c6-b8fa-bd668fcf170e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:50:26.700830 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ad322-e3ef-46c6-b8fa-bd668fcf170e service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:50:26.709049 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1408/2826] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:50:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:50:36.705562 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:51:05.212880 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:51:05.215218 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:05.241320 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012d7fb-bd19-4db8-a275-b2468f3e93a9 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:51:05.248060 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c2738-a700-4fdc-a3f1-cf5ff85c9c13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:51:05.249216 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c2738-a700-4fdc-a3f1-cf5ff85c9c13 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:51:05.249401 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c2738-a700-4fdc-a3f1-cf5ff85c9c13 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:51:05.249772 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c2738-a700-4fdc-a3f1-cf5ff85c9c13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:51:05.251848 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-859c2738-a700-4fdc-a3f1-cf5ff85c9c13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:05.265505 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c2738-a700-4fdc-a3f1-cf5ff85c9c13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:51:05.267364 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c2738-a700-4fdc-a3f1-cf5ff85c9c13 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:51:05.280531 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1419/2827] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:51:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:51:15.274429 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:51:17.601809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad322-e3ef-46c6-b8fa-bd668fcf170e service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:51:17.611628 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ad322-e3ef-46c6-b8fa-bd668fcf170e service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:51:17.615399 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d354a-07c0-4f7f-8af8-fc41c4bfa212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:51:17.616053 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d354a-07c0-4f7f-8af8-fc41c4bfa212 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:51:17.616332 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d354a-07c0-4f7f-8af8-fc41c4bfa212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:51:17.616604 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d354a-07c0-4f7f-8af8-fc41c4bfa212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:51:17.618418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-577d354a-07c0-4f7f-8af8-fc41c4bfa212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:17.688278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d354a-07c0-4f7f-8af8-fc41c4bfa212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:51:17.689715 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d354a-07c0-4f7f-8af8-fc41c4bfa212 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:51:17.699632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1409/2828] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:51:20.806134 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c2738-a700-4fdc-a3f1-cf5ff85c9c13 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:51:20.818638 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c2738-a700-4fdc-a3f1-cf5ff85c9c13 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:51:20.824357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:51:20.825160 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:51:20.825609 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:51:20.825888 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:51:20.827817 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:20.840288 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:51:20.841884 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:51:20.853023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1420/2829] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:51:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:51:23.984451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d354a-07c0-4f7f-8af8-fc41c4bfa212 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:51:24.008484 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d354a-07c0-4f7f-8af8-fc41c4bfa212 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:51:24.016104 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b507fa0-6a54-4439-bc46-34cc2426af9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:51:24.017009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b507fa0-6a54-4439-bc46-34cc2426af9a service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:51:24.017452 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b507fa0-6a54-4439-bc46-34cc2426af9a service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:51:24.017910 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b507fa0-6a54-4439-bc46-34cc2426af9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:51:24.019958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-4b507fa0-6a54-4439-bc46-34cc2426af9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:24.044278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b507fa0-6a54-4439-bc46-34cc2426af9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:51:24.047081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b507fa0-6a54-4439-bc46-34cc2426af9a service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:51:24.062899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:51:24.064599 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:24.075266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1410/2830] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:51:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:51:24.080479 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d17987-f60e-4352-bc17-6771db5bbf0d service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:51:24.085661 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:51:24.086751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:51:24.087222 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:51:24.087847 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] PATH_INFO: `/v3/users/d42b42214d0d46bbb9fe90aed7f5e382` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:51:24.089431 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d42b42214d0d46bbb9fe90aed7f5e382)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:51:24.091586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:51:24.142322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd42b42214d0d46bbb9fe90aed7f5e382'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.142961 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd42b42214d0d46bbb9fe90aed7f5e382'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.151588 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd42b42214d0d46bbb9fe90aed7f5e382'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.163441 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd42b42214d0d46bbb9fe90aed7f5e382'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.164094 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd42b42214d0d46bbb9fe90aed7f5e382'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.176367 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1421/2831] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:51:24 2022] DELETE /identity/v3/users/d42b42214d0d46bbb9fe90aed7f5e382 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:51:24.366772 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b507fa0-6a54-4439-bc46-34cc2426af9a service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:51:24.378260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b507fa0-6a54-4439-bc46-34cc2426af9a service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:51:24.440251 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:51:24.441103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:51:24.441378 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:51:24.441655 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] PATH_INFO: `/v3/projects/01fccbd11a4b4e859bec1a56f42315b2` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:51:24.443089 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01fccbd11a4b4e859bec1a56f42315b2)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:51:24.445073 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:51:24.490518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01fccbd11a4b4e859bec1a56f42315b2', 'request_id': 'req-47cd9128-0035-4ba3-bf75-a69ae96c0f15'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.499136 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01fccbd11a4b4e859bec1a56f42315b2', 'request_id': 'req-47cd9128-0035-4ba3-bf75-a69ae96c0f15'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.505937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01fccbd11a4b4e859bec1a56f42315b2', 'request_id': 'req-47cd9128-0035-4ba3-bf75-a69ae96c0f15'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.516767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] The token cache is being invalidate because project 01fccbd11a4b4e859bec1a56f42315b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:51:24.517051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.517688 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.519253 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1411/2832] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:51:24 2022] DELETE /identity/v3/projects/01fccbd11a4b4e859bec1a56f42315b2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:51:24.523785 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:51:24.525239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:24.538640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca19310-9be7-4fc3-a086-10ca058a422c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:51:24.597053 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:51:24.597958 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:51:24.598233 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:51:24.598493 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] PATH_INFO: `/v3/users/0a41036331c84530833af5c08cac988e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:51:24.599890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a41036331c84530833af5c08cac988e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:51:24.601941 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:51:24.651359 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a41036331c84530833af5c08cac988e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.652009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a41036331c84530833af5c08cac988e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.661009 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a41036331c84530833af5c08cac988e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.666232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:51:24.675310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a41036331c84530833af5c08cac988e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.676239 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a41036331c84530833af5c08cac988e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:24.689787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1422/2833] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:51:24 2022] DELETE /identity/v3/users/0a41036331c84530833af5c08cac988e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:51:24.877656 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:51:24.879157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:24.892650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd9128-0035-4ba3-bf75-a69ae96c0f15 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:51:24.901353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:51:24.954582 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:51:24.955498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:51:24.955782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:51:24.956063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] PATH_INFO: `/v3/projects/ad1140025dba4bc184223e702927ad88` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:51:24.957260 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1140025dba4bc184223e702927ad88)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:51:24.959189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:51:25.017330 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1140025dba4bc184223e702927ad88', 'request_id': 'req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:25.025398 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1140025dba4bc184223e702927ad88', 'request_id': 'req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:25.031890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1140025dba4bc184223e702927ad88', 'request_id': 'req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:25.042684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] The token cache is being invalidate because project ad1140025dba4bc184223e702927ad88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:51:25.042972 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:25.043617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:51:25.045190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1412/2834] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:51:24 2022] DELETE /identity/v3/projects/ad1140025dba4bc184223e702927ad88 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:51:34.686561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:51:35.039560 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:51:45.736769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:51:45.738540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:45.818724 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884dbe3-6b08-4612-8daf-5c988580aaea admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:51:45.886380 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:51:45.887099 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:51:45.887413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:51:45.887691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:51:45.889812 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:51:45.961584 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:51:45.963215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:51:45.973066 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1423/2835] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:51:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:51:55.968598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:52:16.821887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:16.823468 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:16.887881 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5f13e-496c-42ac-81ae-9d3a50076bf2 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:16.892702 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0029-24bc-4fe5-b208-39427ed03e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:16.893477 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0029-24bc-4fe5-b208-39427ed03e02 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:16.893902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0029-24bc-4fe5-b208-39427ed03e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:16.894184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b0029-24bc-4fe5-b208-39427ed03e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:16.896190 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-ef3b0029-24bc-4fe5-b208-39427ed03e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:16.963208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b0029-24bc-4fe5-b208-39427ed03e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:16.964788 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b0029-24bc-4fe5-b208-39427ed03e02 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:16.975212 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1413/2836] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:52:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:52:20.252962 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:20.254768 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:20.269340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a01bd-8df1-43e6-96cf-8f1e78704d39 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:20.274246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:20.275483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:20.275787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:20.276079 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] PATH_INFO: `/v3/users/e80e09005b7e4cdc8f42be866dbd91bb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:20.277391 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e80e09005b7e4cdc8f42be866dbd91bb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:20.279435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:20.330249 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e80e09005b7e4cdc8f42be866dbd91bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.330922 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e80e09005b7e4cdc8f42be866dbd91bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.339375 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e80e09005b7e4cdc8f42be866dbd91bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.351545 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e80e09005b7e4cdc8f42be866dbd91bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.352186 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ecfe2bb3-4156-4962-ad00-9b63ac80f205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e80e09005b7e4cdc8f42be866dbd91bb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.364657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1424/2837] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:52:20 2022] DELETE /identity/v3/users/e80e09005b7e4cdc8f42be866dbd91bb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:52:20.548421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0029-24bc-4fe5-b208-39427ed03e02 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:20.561317 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b0029-24bc-4fe5-b208-39427ed03e02 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:20.617572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:20.618436 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:20.618717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:20.618990 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] PATH_INFO: `/v3/projects/0665267ba574490cbcf958be4d793dd3` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:20.620244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0665267ba574490cbcf958be4d793dd3)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:20.622162 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:20.669706 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0665267ba574490cbcf958be4d793dd3', 'request_id': 'req-9b85a2af-6db6-4821-9b20-04287841a153'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.672163 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:52:20.678843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0665267ba574490cbcf958be4d793dd3', 'request_id': 'req-9b85a2af-6db6-4821-9b20-04287841a153'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.686713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0665267ba574490cbcf958be4d793dd3', 'request_id': 'req-9b85a2af-6db6-4821-9b20-04287841a153'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.702135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] The token cache is being invalidate because project 0665267ba574490cbcf958be4d793dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:52:20.702500 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.703476 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-9b85a2af-6db6-4821-9b20-04287841a153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:20.705866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1414/2838] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:52:20 2022] DELETE /identity/v3/projects/0665267ba574490cbcf958be4d793dd3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:52:20.712963 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe9c8f-8df3-4b03-8c47-2388dcd5abf0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:20.713259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe9c8f-8df3-4b03-8c47-2388dcd5abf0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:20.713542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe9c8f-8df3-4b03-8c47-2388dcd5abf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:20.741873 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-22fe9c8f-8df3-4b03-8c47-2388dcd5abf0 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:52:20.743088 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-22fe9c8f-8df3-4b03-8c47-2388dcd5abf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:20.804199 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1425/2839] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:52:20.812975 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804912f-e5a2-4850-bd2c-cc21269531b6 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:20.813261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804912f-e5a2-4850-bd2c-cc21269531b6 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:20.813498 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804912f-e5a2-4850-bd2c-cc21269531b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:20.833202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d804912f-e5a2-4850-bd2c-cc21269531b6 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:52:20.834418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d804912f-e5a2-4850-bd2c-cc21269531b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:20.843379 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1415/2840] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:52:20.847714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe9c8f-8df3-4b03-8c47-2388dcd5abf0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:20.857524 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe9c8f-8df3-4b03-8c47-2388dcd5abf0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:20.863372 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310001-f469-417e-9742-3eb6fe3d8a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:20.864044 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310001-f469-417e-9742-3eb6fe3d8a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:20.864323 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310001-f469-417e-9742-3eb6fe3d8a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:20.864598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a310001-f469-417e-9742-3eb6fe3d8a3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:20.865374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310001-f469-417e-9742-3eb6fe3d8a3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:52:20.865813 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310001-f469-417e-9742-3eb6fe3d8a3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:20.867532 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a310001-f469-417e-9742-3eb6fe3d8a3b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:20.879578 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1426/2841] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:52:20.883838 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804912f-e5a2-4850-bd2c-cc21269531b6 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:20.893577 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804912f-e5a2-4850-bd2c-cc21269531b6 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:20.901900 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ec1b9-9e49-42c9-ab7f-abe4746dd3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:20.902798 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ec1b9-9e49-42c9-ab7f-abe4746dd3be admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:20.903137 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ec1b9-9e49-42c9-ab7f-abe4746dd3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:20.903563 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ec1b9-9e49-42c9-ab7f-abe4746dd3be admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:20.904567 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ec1b9-9e49-42c9-ab7f-abe4746dd3be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:20.907360 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ec1b9-9e49-42c9-ab7f-abe4746dd3be admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:20.947889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1416/2842] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:52:20 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:52:20.952226 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310001-f469-417e-9742-3eb6fe3d8a3b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:20.963225 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a310001-f469-417e-9742-3eb6fe3d8a3b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:21.011540 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:52:21.023998 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e47f6b-7743-4a21-827f-bd76a996f4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:21.024688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e47f6b-7743-4a21-827f-bd76a996f4ab admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:21.024979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e47f6b-7743-4a21-827f-bd76a996f4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:21.025263 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e47f6b-7743-4a21-827f-bd76a996f4ab admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:21.025976 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e47f6b-7743-4a21-827f-bd76a996f4ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:21.028292 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e47f6b-7743-4a21-827f-bd76a996f4ab admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:21.052273 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1427/2843] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:52:20 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:52:21.057818 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ec1b9-9e49-42c9-ab7f-abe4746dd3be admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:21.068650 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ec1b9-9e49-42c9-ab7f-abe4746dd3be admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:21.073533 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45308d80-197f-4a1f-9785-49d2baba07f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:21.074189 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45308d80-197f-4a1f-9785-49d2baba07f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:21.074466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45308d80-197f-4a1f-9785-49d2baba07f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:21.074737 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45308d80-197f-4a1f-9785-49d2baba07f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:21.075482 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45308d80-197f-4a1f-9785-49d2baba07f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:21.077081 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45308d80-197f-4a1f-9785-49d2baba07f4 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:21.087255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1417/2844] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:52:21.091331 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e47f6b-7743-4a21-827f-bd76a996f4ab admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:21.104246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e47f6b-7743-4a21-827f-bd76a996f4ab admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:21.109419 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f07539-1340-485f-a483-304531b7959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:21.110101 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07539-1340-485f-a483-304531b7959d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:21.110386 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07539-1340-485f-a483-304531b7959d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:21.110660 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f07539-1340-485f-a483-304531b7959d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:21.111551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f07539-1340-485f-a483-304531b7959d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:21.114690 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f07539-1340-485f-a483-304531b7959d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:21.129046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1428/2845] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:52:21.133107 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45308d80-197f-4a1f-9785-49d2baba07f4 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:21.144548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45308d80-197f-4a1f-9785-49d2baba07f4 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:21.150440 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9afba-8d2c-4137-bf8b-832634028ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:21.151376 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9afba-8d2c-4137-bf8b-832634028ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:21.151652 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9afba-8d2c-4137-bf8b-832634028ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:21.151928 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9afba-8d2c-4137-bf8b-832634028ee8 admin admin] PATH_INFO: `/v3/projects/c2606e542ea74559be1d0009dc249b87/users/13631faf0a06418887315f17b58a8574/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:21.168568 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9afba-8d2c-4137-bf8b-832634028ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2606e542ea74559be1d0009dc249b87, user_id=13631faf0a06418887315f17b58a8574, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:21.171343 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9afba-8d2c-4137-bf8b-832634028ee8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:21.187767 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1418/2846] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:52:21 2022] PUT /identity/v3/projects/c2606e542ea74559be1d0009dc249b87/users/13631faf0a06418887315f17b58a8574/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:52:21.199430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dd72e-de36-419b-9c4c-a7e504463541 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:21.199734 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dd72e-de36-419b-9c4c-a7e504463541 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:21.200016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6dd72e-de36-419b-9c4c-a7e504463541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:21.231892 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4a6dd72e-de36-419b-9c4c-a7e504463541 None None] MFA Rules not processed for user `13631faf0a06418887315f17b58a8574`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:52:21.232422 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4a6dd72e-de36-419b-9c4c-a7e504463541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:21.299541 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1429/2847] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:52:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:52:21.311242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9afba-8d2c-4137-bf8b-832634028ee8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:21.313153 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-efb9afba-8d2c-4137-bf8b-832634028ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:21.398713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9afba-8d2c-4137-bf8b-832634028ee8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:21.403184 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7e89d-9d7a-4336-8d7c-8110b8ce4281 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:21.403859 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7e89d-9d7a-4336-8d7c-8110b8ce4281 service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:21.404161 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7e89d-9d7a-4336-8d7c-8110b8ce4281 service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:21.404392 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7e89d-9d7a-4336-8d7c-8110b8ce4281 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:21.414310 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7e89d-9d7a-4336-8d7c-8110b8ce4281 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:21.415787 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7e89d-9d7a-4336-8d7c-8110b8ce4281 service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:21.425774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1419/2848] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:52:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:52:31.296598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:52:31.421064 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:DELETE=3,keystone:SELECT=72,keystone:INSERT=2 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:52:45.423253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6dd72e-de36-419b-9c4c-a7e504463541 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:45.488556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6dd72e-de36-419b-9c4c-a7e504463541 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:45.498227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:45.499522 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:45.499877 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:45.500224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] PATH_INFO: `/v3/users/13631faf0a06418887315f17b58a8574` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:45.502238 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13631faf0a06418887315f17b58a8574)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:45.504280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:45.555118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13631faf0a06418887315f17b58a8574'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:45.555876 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13631faf0a06418887315f17b58a8574'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:45.564390 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13631faf0a06418887315f17b58a8574'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:45.576130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13631faf0a06418887315f17b58a8574'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:45.576765 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13631faf0a06418887315f17b58a8574'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:45.589981 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1430/2849] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:52:45 2022] DELETE /identity/v3/users/13631faf0a06418887315f17b58a8574 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:52:45.594894 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272753d-0800-4571-84df-d63f938ae48b None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:45.595457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272753d-0800-4571-84df-d63f938ae48b None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:45.595457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272753d-0800-4571-84df-d63f938ae48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:45.613995 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-7272753d-0800-4571-84df-d63f938ae48b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:52:45.615207 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7272753d-0800-4571-84df-d63f938ae48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:45.677913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1420/2850] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:52:45.687737 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:45.689725 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:45.751745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fdb34c-98b6-43e6-92b3-12af6aeb7af6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:45.756181 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66c44d-ac70-4ad7-b86f-18963d3b2d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:45.756836 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66c44d-ac70-4ad7-b86f-18963d3b2d14 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:45.757117 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66c44d-ac70-4ad7-b86f-18963d3b2d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:45.757409 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66c44d-ac70-4ad7-b86f-18963d3b2d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:45.768322 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66c44d-ac70-4ad7-b86f-18963d3b2d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:45.770666 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66c44d-ac70-4ad7-b86f-18963d3b2d14 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:45.787326 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1431/2851] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:52:46.086030 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272753d-0800-4571-84df-d63f938ae48b None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:46.087632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-7272753d-0800-4571-84df-d63f938ae48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:46.099966 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272753d-0800-4571-84df-d63f938ae48b None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:46.105171 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:46.106197 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.106561 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.106901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] PATH_INFO: `/v3/projects/c2606e542ea74559be1d0009dc249b87` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.110009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2606e542ea74559be1d0009dc249b87)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:46.112051 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:46.160850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2606e542ea74559be1d0009dc249b87', 'request_id': 'req-0924aa6a-7162-4d11-978e-72b138cfd6dc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:46.169669 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2606e542ea74559be1d0009dc249b87', 'request_id': 'req-0924aa6a-7162-4d11-978e-72b138cfd6dc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:46.176801 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2606e542ea74559be1d0009dc249b87', 'request_id': 'req-0924aa6a-7162-4d11-978e-72b138cfd6dc'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:46.187832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] The token cache is being invalidate because project c2606e542ea74559be1d0009dc249b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:52:46.188128 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:46.188887 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-0924aa6a-7162-4d11-978e-72b138cfd6dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:52:46.190632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1421/2852] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:52:46 2022] DELETE /identity/v3/projects/c2606e542ea74559be1d0009dc249b87 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:52:46.195899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793541f6-a468-4c6a-8ed9-0ed655f97645 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.196187 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793541f6-a468-4c6a-8ed9-0ed655f97645 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.196466 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793541f6-a468-4c6a-8ed9-0ed655f97645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.214769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-793541f6-a468-4c6a-8ed9-0ed655f97645 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:52:46.215972 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-793541f6-a468-4c6a-8ed9-0ed655f97645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:46.234054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:52:46.282267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1432/2853] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:52:46.290518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611dfe92-8db1-4558-a320-2fbc09b6f40c None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.290828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611dfe92-8db1-4558-a320-2fbc09b6f40c None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.291548 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611dfe92-8db1-4558-a320-2fbc09b6f40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.311491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-611dfe92-8db1-4558-a320-2fbc09b6f40c None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:52:46.312693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-611dfe92-8db1-4558-a320-2fbc09b6f40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:46.321741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1422/2854] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:52:46.326217 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793541f6-a468-4c6a-8ed9-0ed655f97645 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:46.336845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793541f6-a468-4c6a-8ed9-0ed655f97645 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:46.342586 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280352d-2ae5-4cea-9580-b3332dd17cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:46.343253 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280352d-2ae5-4cea-9580-b3332dd17cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.343572 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280352d-2ae5-4cea-9580-b3332dd17cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.343845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280352d-2ae5-4cea-9580-b3332dd17cd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.344623 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280352d-2ae5-4cea-9580-b3332dd17cd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:52:46.344948 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280352d-2ae5-4cea-9580-b3332dd17cd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:46.346665 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280352d-2ae5-4cea-9580-b3332dd17cd7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:46.358715 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1433/2855] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:52:46.364063 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611dfe92-8db1-4558-a320-2fbc09b6f40c None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:46.378278 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611dfe92-8db1-4558-a320-2fbc09b6f40c None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:46.386823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd434b0a-5d47-4b56-a77e-546deb1cb5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:46.387815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd434b0a-5d47-4b56-a77e-546deb1cb5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.388164 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd434b0a-5d47-4b56-a77e-546deb1cb5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.388441 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd434b0a-5d47-4b56-a77e-546deb1cb5c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.389437 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd434b0a-5d47-4b56-a77e-546deb1cb5c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:46.392266 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd434b0a-5d47-4b56-a77e-546deb1cb5c8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:46.425371 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1423/2856] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:52:46 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 02 03:52:46.429592 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280352d-2ae5-4cea-9580-b3332dd17cd7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:46.440694 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280352d-2ae5-4cea-9580-b3332dd17cd7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:46.498062 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202648e-4de1-4185-bd4b-53bcdb6d0154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:46.498726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202648e-4de1-4185-bd4b-53bcdb6d0154 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.499013 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202648e-4de1-4185-bd4b-53bcdb6d0154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.499280 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202648e-4de1-4185-bd4b-53bcdb6d0154 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.499999 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202648e-4de1-4185-bd4b-53bcdb6d0154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:46.502172 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202648e-4de1-4185-bd4b-53bcdb6d0154 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:46.528194 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1434/2857] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:52:46 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:52:46.532505 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd434b0a-5d47-4b56-a77e-546deb1cb5c8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:46.543620 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd434b0a-5d47-4b56-a77e-546deb1cb5c8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:46.549823 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90328bf-37a6-461b-8452-b57d58167665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:46.550867 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90328bf-37a6-461b-8452-b57d58167665 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.551271 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90328bf-37a6-461b-8452-b57d58167665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.551716 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90328bf-37a6-461b-8452-b57d58167665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.552882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90328bf-37a6-461b-8452-b57d58167665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:46.555479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90328bf-37a6-461b-8452-b57d58167665 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:46.574037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1424/2858] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:52:46.577852 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202648e-4de1-4185-bd4b-53bcdb6d0154 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:46.588534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202648e-4de1-4185-bd4b-53bcdb6d0154 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:46.594016 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba88bc2-ad77-4c90-b87a-c5dd72578a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:46.594712 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba88bc2-ad77-4c90-b87a-c5dd72578a22 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.594996 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba88bc2-ad77-4c90-b87a-c5dd72578a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.595453 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba88bc2-ad77-4c90-b87a-c5dd72578a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.596180 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba88bc2-ad77-4c90-b87a-c5dd72578a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:46.597799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba88bc2-ad77-4c90-b87a-c5dd72578a22 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:46.608074 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1435/2859] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:52:46.612423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90328bf-37a6-461b-8452-b57d58167665 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:46.622797 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90328bf-37a6-461b-8452-b57d58167665 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:46.628460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b08862-047c-449b-9282-eabd88f6754a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:46.629336 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b08862-047c-449b-9282-eabd88f6754a admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.629624 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b08862-047c-449b-9282-eabd88f6754a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.629891 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b08862-047c-449b-9282-eabd88f6754a admin admin] PATH_INFO: `/v3/projects/b5bba64cb7594693b4f59dfd2d6363a8/users/0dc20b90b81c4fb49c0aa43386292aeb/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.644721 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b08862-047c-449b-9282-eabd88f6754a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bba64cb7594693b4f59dfd2d6363a8, user_id=0dc20b90b81c4fb49c0aa43386292aeb, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:46.647356 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b08862-047c-449b-9282-eabd88f6754a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:46.662785 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1425/2860] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:52:46 2022] PUT /identity/v3/projects/b5bba64cb7594693b4f59dfd2d6363a8/users/0dc20b90b81c4fb49c0aa43386292aeb/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:52:46.674943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911080e-39b1-4027-8f4d-0a0991789ad9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.675232 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911080e-39b1-4027-8f4d-0a0991789ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.675612 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911080e-39b1-4027-8f4d-0a0991789ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.706325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9911080e-39b1-4027-8f4d-0a0991789ad9 None None] MFA Rules not processed for user `0dc20b90b81c4fb49c0aa43386292aeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:52:46.708412 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9911080e-39b1-4027-8f4d-0a0991789ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:46.789054 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1436/2861] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:52:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:52:46.801215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b08862-047c-449b-9282-eabd88f6754a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:52:46.802967 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-42b08862-047c-449b-9282-eabd88f6754a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:52:46.826622 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:52:46.865327 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b08862-047c-449b-9282-eabd88f6754a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:52:46.869514 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b1c5f6-9711-4ecb-8891-7e6396abcd95 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:52:46.870167 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1c5f6-9711-4ecb-8891-7e6396abcd95 service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:52:46.870451 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1c5f6-9711-4ecb-8891-7e6396abcd95 service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:52:46.870729 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b1c5f6-9711-4ecb-8891-7e6396abcd95 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:52:46.879687 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1c5f6-9711-4ecb-8891-7e6396abcd95 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:52:46.881140 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b1c5f6-9711-4ecb-8891-7e6396abcd95 service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:52:46.890200 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1426/2862] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:52:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:52:56.785899 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:52:56.886203 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=17 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:53:27.786633 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911080e-39b1-4027-8f4d-0a0991789ad9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:27.853616 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911080e-39b1-4027-8f4d-0a0991789ad9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:27.858605 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:27.859507 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:27.859783 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:27.860055 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] PATH_INFO: `/v3/users/0dc20b90b81c4fb49c0aa43386292aeb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:27.861282 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dc20b90b81c4fb49c0aa43386292aeb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:27.863229 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:27.915747 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dc20b90b81c4fb49c0aa43386292aeb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:27.916751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dc20b90b81c4fb49c0aa43386292aeb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:27.926243 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dc20b90b81c4fb49c0aa43386292aeb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:27.937979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dc20b90b81c4fb49c0aa43386292aeb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:27.938640 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dc20b90b81c4fb49c0aa43386292aeb'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:27.950755 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1437/2863] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:53:27 2022] DELETE /identity/v3/users/0dc20b90b81c4fb49c0aa43386292aeb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:53:27.955587 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3603b6-56f4-49e7-a3b9-5b86cef9f013 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:27.956103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3603b6-56f4-49e7-a3b9-5b86cef9f013 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:27.956103 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3603b6-56f4-49e7-a3b9-5b86cef9f013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:27.973667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-9a3603b6-56f4-49e7-a3b9-5b86cef9f013 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:53:27.975120 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9a3603b6-56f4-49e7-a3b9-5b86cef9f013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:53:28.037843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1427/2864] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:53:28.049215 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:28.051224 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:53:28.114096 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d262b-91c5-4eae-908b-bebfef6397d1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:28.119796 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1bbda-f706-49bd-a38c-b6b9b9ead109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:28.120465 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1bbda-f706-49bd-a38c-b6b9b9ead109 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.120742 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1bbda-f706-49bd-a38c-b6b9b9ead109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.121034 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1bbda-f706-49bd-a38c-b6b9b9ead109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.131072 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1bbda-f706-49bd-a38c-b6b9b9ead109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:28.132542 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1bbda-f706-49bd-a38c-b6b9b9ead109 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:28.142188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1438/2865] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:53:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:53:28.418741 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3603b6-56f4-49e7-a3b9-5b86cef9f013 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:28.420215 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-9a3603b6-56f4-49e7-a3b9-5b86cef9f013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:53:28.432491 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3603b6-56f4-49e7-a3b9-5b86cef9f013 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:28.437949 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:28.438828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.439108 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.439420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] PATH_INFO: `/v3/projects/b5bba64cb7594693b4f59dfd2d6363a8` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.440714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bba64cb7594693b4f59dfd2d6363a8)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:28.442854 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:28.493759 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bba64cb7594693b4f59dfd2d6363a8', 'request_id': 'req-8adbe61e-681c-4b38-8322-8152e106b126'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:28.501471 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bba64cb7594693b4f59dfd2d6363a8', 'request_id': 'req-8adbe61e-681c-4b38-8322-8152e106b126'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:28.507901 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bba64cb7594693b4f59dfd2d6363a8', 'request_id': 'req-8adbe61e-681c-4b38-8322-8152e106b126'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:28.517866 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] The token cache is being invalidate because project b5bba64cb7594693b4f59dfd2d6363a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:53:28.518157 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:28.518829 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-8adbe61e-681c-4b38-8322-8152e106b126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:53:28.520470 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1428/2866] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:53:28 2022] DELETE /identity/v3/projects/b5bba64cb7594693b4f59dfd2d6363a8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:53:28.525484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f5595-b844-44f7-9a83-35573286131d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.525769 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f5595-b844-44f7-9a83-35573286131d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.526042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f5595-b844-44f7-9a83-35573286131d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.545030 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a96f5595-b844-44f7-9a83-35573286131d None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:53:28.546237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a96f5595-b844-44f7-9a83-35573286131d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:53:28.564426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:53:28.608943 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1439/2867] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:53:28.616912 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe6f52-8c30-4f36-bb35-2c5bc973d3ec None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.617202 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe6f52-8c30-4f36-bb35-2c5bc973d3ec None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.617473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe6f52-8c30-4f36-bb35-2c5bc973d3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.637118 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-0fbe6f52-8c30-4f36-bb35-2c5bc973d3ec None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:53:28.638334 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-0fbe6f52-8c30-4f36-bb35-2c5bc973d3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:53:28.648546 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1429/2868] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:53:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:53:28.653298 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f5595-b844-44f7-9a83-35573286131d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:28.663773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f5595-b844-44f7-9a83-35573286131d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:28.669650 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b15cab-e407-42a2-9bf7-97a71d334f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:28.670337 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b15cab-e407-42a2-9bf7-97a71d334f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.670602 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b15cab-e407-42a2-9bf7-97a71d334f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.670881 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b15cab-e407-42a2-9bf7-97a71d334f93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.671639 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b15cab-e407-42a2-9bf7-97a71d334f93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:53:28.671939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b15cab-e407-42a2-9bf7-97a71d334f93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:28.673547 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b15cab-e407-42a2-9bf7-97a71d334f93 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:28.685472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1440/2869] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:53:28.689865 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe6f52-8c30-4f36-bb35-2c5bc973d3ec None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:28.699217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe6f52-8c30-4f36-bb35-2c5bc973d3ec None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:28.704617 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8062c-f7c5-4041-90b3-4f0a1fc04846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:28.705261 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8062c-f7c5-4041-90b3-4f0a1fc04846 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.705545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8062c-f7c5-4041-90b3-4f0a1fc04846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.705825 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8062c-f7c5-4041-90b3-4f0a1fc04846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.706510 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8062c-f7c5-4041-90b3-4f0a1fc04846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:28.708518 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8062c-f7c5-4041-90b3-4f0a1fc04846 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:28.736605 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1430/2870] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:53:28 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:53:28.741010 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b15cab-e407-42a2-9bf7-97a71d334f93 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:28.751486 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b15cab-e407-42a2-9bf7-97a71d334f93 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:28.810070 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a1213-7a91-4a02-af2d-f8516c9b2019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:28.810758 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a1213-7a91-4a02-af2d-f8516c9b2019 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.811001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a1213-7a91-4a02-af2d-f8516c9b2019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.811276 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a1213-7a91-4a02-af2d-f8516c9b2019 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.811991 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a1213-7a91-4a02-af2d-f8516c9b2019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:28.813921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a1213-7a91-4a02-af2d-f8516c9b2019 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:28.837285 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1441/2871] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:53:28 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:53:28.841664 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8062c-f7c5-4041-90b3-4f0a1fc04846 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:28.853112 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8062c-f7c5-4041-90b3-4f0a1fc04846 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:28.858799 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64846293-2639-4a8a-a5d5-6076685aa5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:28.859527 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64846293-2639-4a8a-a5d5-6076685aa5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.859815 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64846293-2639-4a8a-a5d5-6076685aa5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.860092 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64846293-2639-4a8a-a5d5-6076685aa5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.860817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64846293-2639-4a8a-a5d5-6076685aa5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:28.862497 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64846293-2639-4a8a-a5d5-6076685aa5a8 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:28.873761 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1431/2872] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:53:28.879921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a1213-7a91-4a02-af2d-f8516c9b2019 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:28.893726 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a1213-7a91-4a02-af2d-f8516c9b2019 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:28.900078 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45046d3d-ba06-48d3-aeb2-56cafba7901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:28.900802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45046d3d-ba06-48d3-aeb2-56cafba7901b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.901084 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45046d3d-ba06-48d3-aeb2-56cafba7901b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.901360 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45046d3d-ba06-48d3-aeb2-56cafba7901b admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.902090 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45046d3d-ba06-48d3-aeb2-56cafba7901b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:28.903751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45046d3d-ba06-48d3-aeb2-56cafba7901b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:28.914628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1442/2873] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:53:28.919809 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64846293-2639-4a8a-a5d5-6076685aa5a8 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:28.931597 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64846293-2639-4a8a-a5d5-6076685aa5a8 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:28.936784 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788978af-9404-4bac-9a46-b7597c060b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:28.937822 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788978af-9404-4bac-9a46-b7597c060b60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.938165 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788978af-9404-4bac-9a46-b7597c060b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.938421 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788978af-9404-4bac-9a46-b7597c060b60 admin admin] PATH_INFO: `/v3/projects/4fcd40e73e6646a4b098a6f56e8bf135/users/7a8763187e5f4b2588b1250beb981526/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:28.955135 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788978af-9404-4bac-9a46-b7597c060b60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fcd40e73e6646a4b098a6f56e8bf135, user_id=7a8763187e5f4b2588b1250beb981526, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:28.957795 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788978af-9404-4bac-9a46-b7597c060b60 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:28.973309 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1432/2874] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:53:28 2022] PUT /identity/v3/projects/4fcd40e73e6646a4b098a6f56e8bf135/users/7a8763187e5f4b2588b1250beb981526/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:53:28.984003 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01519293-fc91-43cc-8ad1-7a5eb77405ad None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:28.984312 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01519293-fc91-43cc-8ad1-7a5eb77405ad None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:28.984565 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01519293-fc91-43cc-8ad1-7a5eb77405ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:29.012046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-01519293-fc91-43cc-8ad1-7a5eb77405ad None None] MFA Rules not processed for user `7a8763187e5f4b2588b1250beb981526`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:53:29.013259 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-01519293-fc91-43cc-8ad1-7a5eb77405ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:53:29.081404 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1443/2875] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:53:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:53:29.093321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788978af-9404-4bac-9a46-b7597c060b60 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:29.095155 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-788978af-9404-4bac-9a46-b7597c060b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:53:29.117249 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:53:29.157204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788978af-9404-4bac-9a46-b7597c060b60 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:29.161241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f00e9-9da1-46dd-a192-dcd8af15ad15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:29.161965 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f00e9-9da1-46dd-a192-dcd8af15ad15 service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:29.162384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f00e9-9da1-46dd-a192-dcd8af15ad15 service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:29.162566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f00e9-9da1-46dd-a192-dcd8af15ad15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:29.171663 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f00e9-9da1-46dd-a192-dcd8af15ad15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:29.173113 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f00e9-9da1-46dd-a192-dcd8af15ad15 service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:29.182326 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1433/2876] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:53:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:53:29.265799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01519293-fc91-43cc-8ad1-7a5eb77405ad None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:53:29.267519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-01519293-fc91-43cc-8ad1-7a5eb77405ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:53:29.289198 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:53:29.343652 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01519293-fc91-43cc-8ad1-7a5eb77405ad None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:53:29.349364 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56652aff-b5f2-4d74-9bcf-e9dc0f427621 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:53:29.350906 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56652aff-b5f2-4d74-9bcf-e9dc0f427621 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:53:29.351537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56652aff-b5f2-4d74-9bcf-e9dc0f427621 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:53:29.351818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56652aff-b5f2-4d74-9bcf-e9dc0f427621 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:53:29.353879 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-56652aff-b5f2-4d74-9bcf-e9dc0f427621 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:53:29.366802 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56652aff-b5f2-4d74-9bcf-e9dc0f427621 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:53:29.368463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56652aff-b5f2-4d74-9bcf-e9dc0f427621 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:53:29.380845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1444/2877] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:53:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:53:39.178244 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=17 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:53:39.375104 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=24 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:54:02.363255 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f00e9-9da1-46dd-a192-dcd8af15ad15 service glance] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:02.441779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f00e9-9da1-46dd-a192-dcd8af15ad15 service glance] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:02.448574 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:02.449457 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:02.449735 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:02.450008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] PATH_INFO: `/v3/users/7a8763187e5f4b2588b1250beb981526` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:02.451423 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8763187e5f4b2588b1250beb981526)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:02.453445 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:02.502684 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8763187e5f4b2588b1250beb981526'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:02.503359 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8763187e5f4b2588b1250beb981526'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:02.511873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8763187e5f4b2588b1250beb981526'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:02.512499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8763187e5f4b2588b1250beb981526'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:02.524462 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8763187e5f4b2588b1250beb981526'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:02.537699 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1434/2878] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:54:02 2022] DELETE /identity/v3/users/7a8763187e5f4b2588b1250beb981526 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:54:02.543428 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e58c3-fc46-481c-894c-743ce52c141a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:02.543979 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e58c3-fc46-481c-894c-743ce52c141a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:02.544087 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e58c3-fc46-481c-894c-743ce52c141a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:02.561324 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bf8e58c3-fc46-481c-894c-743ce52c141a None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:54:02.562552 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bf8e58c3-fc46-481c-894c-743ce52c141a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:02.621435 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1445/2879] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:54:02.631182 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:02.632782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:02.691204 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31f79a-4e30-483f-9039-1cafba050526 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:02.695507 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27869f-3ee0-4424-a952-28caccffefd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:02.696145 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27869f-3ee0-4424-a952-28caccffefd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:02.696424 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27869f-3ee0-4424-a952-28caccffefd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:02.696693 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27869f-3ee0-4424-a952-28caccffefd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:02.706607 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27869f-3ee0-4424-a952-28caccffefd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:02.708078 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27869f-3ee0-4424-a952-28caccffefd3 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:02.717529 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1435/2880] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:54:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:54:03.003556 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e58c3-fc46-481c-894c-743ce52c141a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:03.005228 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bf8e58c3-fc46-481c-894c-743ce52c141a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:03.018455 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e58c3-fc46-481c-894c-743ce52c141a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:03.024384 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:03.025387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.025676 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.025951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] PATH_INFO: `/v3/projects/4fcd40e73e6646a4b098a6f56e8bf135` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.027310 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fcd40e73e6646a4b098a6f56e8bf135)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:03.029278 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:03.084446 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fcd40e73e6646a4b098a6f56e8bf135', 'request_id': 'req-8a37274c-b05d-4557-80d1-703f043986cc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:03.090227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fcd40e73e6646a4b098a6f56e8bf135', 'request_id': 'req-8a37274c-b05d-4557-80d1-703f043986cc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:03.096745 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fcd40e73e6646a4b098a6f56e8bf135', 'request_id': 'req-8a37274c-b05d-4557-80d1-703f043986cc'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:03.111714 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] The token cache is being invalidate because project 4fcd40e73e6646a4b098a6f56e8bf135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:54:03.112002 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:03.112624 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8a37274c-b05d-4557-80d1-703f043986cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:54:03.114246 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1446/2881] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:54:03 2022] DELETE /identity/v3/projects/4fcd40e73e6646a4b098a6f56e8bf135 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:54:03.119526 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e72a7b-22ad-4430-9c58-05c0f7e1bca8 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.120000 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e72a7b-22ad-4430-9c58-05c0f7e1bca8 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.120281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e72a7b-22ad-4430-9c58-05c0f7e1bca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.139147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-f1e72a7b-22ad-4430-9c58-05c0f7e1bca8 None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:54:03.140311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-f1e72a7b-22ad-4430-9c58-05c0f7e1bca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:03.159088 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:54:03.211850 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1436/2882] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 02 03:54:03.220122 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08082d2f-e882-40c2-878b-33a26f81324b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.220416 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08082d2f-e882-40c2-878b-33a26f81324b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.220693 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08082d2f-e882-40c2-878b-33a26f81324b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.239845 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-08082d2f-e882-40c2-878b-33a26f81324b None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:54:03.241037 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08082d2f-e882-40c2-878b-33a26f81324b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:03.254157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1447/2883] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:54:03.259047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e72a7b-22ad-4430-9c58-05c0f7e1bca8 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:03.269479 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e72a7b-22ad-4430-9c58-05c0f7e1bca8 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:03.274871 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fc952-a6ec-4308-b863-9aa8794c811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:03.275615 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fc952-a6ec-4308-b863-9aa8794c811f admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.275884 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fc952-a6ec-4308-b863-9aa8794c811f admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.276129 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fc952-a6ec-4308-b863-9aa8794c811f admin admin] PATH_INFO: `/v3/domains` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.276882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fc952-a6ec-4308-b863-9aa8794c811f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80354) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 02 03:54:03.277186 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fc952-a6ec-4308-b863-9aa8794c811f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:03.278792 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fc952-a6ec-4308-b863-9aa8794c811f admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:03.290455 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1437/2884] 10.222.0.42 () {58 vars in 1152 bytes} [Sat Jul 2 03:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 02 03:54:03.295821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08082d2f-e882-40c2-878b-33a26f81324b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:03.308910 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08082d2f-e882-40c2-878b-33a26f81324b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:03.314939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a262-ef1b-4931-b7d3-bf7907816138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:03.315628 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a262-ef1b-4931-b7d3-bf7907816138 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.315911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a262-ef1b-4931-b7d3-bf7907816138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.316182 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704a262-ef1b-4931-b7d3-bf7907816138 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.316869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704a262-ef1b-4931-b7d3-bf7907816138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:03.318821 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b704a262-ef1b-4931-b7d3-bf7907816138 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:03.346691 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1448/2885] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:54:03 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:54:03.351008 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fc952-a6ec-4308-b863-9aa8794c811f admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:03.362534 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fc952-a6ec-4308-b863-9aa8794c811f admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:03.424099 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17646595-9288-4750-9366-d9c0f70ff220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:03.424762 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17646595-9288-4750-9366-d9c0f70ff220 admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.424942 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17646595-9288-4750-9366-d9c0f70ff220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.425227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17646595-9288-4750-9366-d9c0f70ff220 admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.425946 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17646595-9288-4750-9366-d9c0f70ff220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:03.427828 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17646595-9288-4750-9366-d9c0f70ff220 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:03.452992 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1438/2886] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:54:03 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:54:03.457237 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a262-ef1b-4931-b7d3-bf7907816138 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:03.468241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704a262-ef1b-4931-b7d3-bf7907816138 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:03.474012 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fa94a-13bb-45b9-8542-dbd539104f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:03.474773 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fa94a-13bb-45b9-8542-dbd539104f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.475057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fa94a-13bb-45b9-8542-dbd539104f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.475325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fa94a-13bb-45b9-8542-dbd539104f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.476157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773fa94a-13bb-45b9-8542-dbd539104f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:03.477781 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773fa94a-13bb-45b9-8542-dbd539104f8e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:03.488701 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1449/2887] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:54:03.492782 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17646595-9288-4750-9366-d9c0f70ff220 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:03.503875 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17646595-9288-4750-9366-d9c0f70ff220 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:03.510350 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439261b5-8a07-40b9-920e-e2a079a7a20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:03.510350 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439261b5-8a07-40b9-920e-e2a079a7a20a admin admin] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.510632 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439261b5-8a07-40b9-920e-e2a079a7a20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.510755 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439261b5-8a07-40b9-920e-e2a079a7a20a admin admin] PATH_INFO: `/v3/roles` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.511473 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439261b5-8a07-40b9-920e-e2a079a7a20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:03.513074 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439261b5-8a07-40b9-920e-e2a079a7a20a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:03.523412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1439/2888] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:54:03.529850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fa94a-13bb-45b9-8542-dbd539104f8e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:03.544315 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fa94a-13bb-45b9-8542-dbd539104f8e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:03.551635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5019828-c5c3-43a5-9379-53a7660e6c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:03.552951 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5019828-c5c3-43a5-9379-53a7660e6c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.553275 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5019828-c5c3-43a5-9379-53a7660e6c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.553615 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5019828-c5c3-43a5-9379-53a7660e6c31 admin admin] PATH_INFO: `/v3/projects/642a93dd7b34496491123f30dce4438d/users/ed4e5bbc53a74f7083525adc968db049/roles/6aab4b825fea4f4292adc656ae17a709` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.570680 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5019828-c5c3-43a5-9379-53a7660e6c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642a93dd7b34496491123f30dce4438d, user_id=ed4e5bbc53a74f7083525adc968db049, role_id=6aab4b825fea4f4292adc656ae17a709)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:03.573519 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5019828-c5c3-43a5-9379-53a7660e6c31 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:03.589351 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1450/2889] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:54:03 2022] PUT /identity/v3/projects/642a93dd7b34496491123f30dce4438d/users/ed4e5bbc53a74f7083525adc968db049/roles/6aab4b825fea4f4292adc656ae17a709 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:54:03.594487 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddec88d-3136-4b4f-9120-9db98862c5bf None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.594779 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddec88d-3136-4b4f-9120-9db98862c5bf None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.595061 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddec88d-3136-4b4f-9120-9db98862c5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.612932 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-6ddec88d-3136-4b4f-9120-9db98862c5bf None None] MFA Rules not processed for user `c350860378554603affd063cfacfc453`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:54:03.614124 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-6ddec88d-3136-4b4f-9120-9db98862c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:03.674038 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1440/2890] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:54:03.684751 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5019828-c5c3-43a5-9379-53a7660e6c31 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:03.686810 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a5019828-c5c3-43a5-9379-53a7660e6c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:03.710270 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:54:03.746855 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5019828-c5c3-43a5-9379-53a7660e6c31 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:03.750787 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40ad3-1c6b-4c1c-8ceb-2129da4a2d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:03.751483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40ad3-1c6b-4c1c-8ceb-2129da4a2d69 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:03.751762 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40ad3-1c6b-4c1c-8ceb-2129da4a2d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:03.752035 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40ad3-1c6b-4c1c-8ceb-2129da4a2d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:03.761878 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40ad3-1c6b-4c1c-8ceb-2129da4a2d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:03.763325 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40ad3-1c6b-4c1c-8ceb-2129da4a2d69 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:03.772850 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1451/2891] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:54:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:54:07.711531 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d19162-8108-4cc5-8a80-8d897e868d47 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:07.712990 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d19162-8108-4cc5-8a80-8d897e868d47 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:07.713409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d19162-8108-4cc5-8a80-8d897e868d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:07.751609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-d5d19162-8108-4cc5-8a80-8d897e868d47 None None] MFA Rules not processed for user `ed4e5bbc53a74f7083525adc968db049`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:54:07.753418 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-d5d19162-8108-4cc5-8a80-8d897e868d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:07.776937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:54:07.838847 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1441/2892] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:54:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:54:07.843902 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40ad3-1c6b-4c1c-8ceb-2129da4a2d69 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:07.855098 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40ad3-1c6b-4c1c-8ceb-2129da4a2d69 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:07.861101 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443df08-3159-4816-8135-3d47819a6e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:07.861984 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443df08-3159-4816-8135-3d47819a6e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:07.862168 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443df08-3159-4816-8135-3d47819a6e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:07.863809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443df08-3159-4816-8135-3d47819a6e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:07.863809 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443df08-3159-4816-8135-3d47819a6e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:07.865154 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443df08-3159-4816-8135-3d47819a6e3e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:07.903188 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1452/2893] 10.222.0.42 () {60 vars in 1152 bytes} [Sat Jul 2 03:54:07 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:54:07.907513 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d19162-8108-4cc5-8a80-8d897e868d47 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:07.921640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d19162-8108-4cc5-8a80-8d897e868d47 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:08.001821 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc04201-31f4-49f7-bdb8-14dffc43037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:08.002488 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc04201-31f4-49f7-bdb8-14dffc43037a admin admin] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:08.002775 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc04201-31f4-49f7-bdb8-14dffc43037a admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:08.003047 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc04201-31f4-49f7-bdb8-14dffc43037a admin admin] PATH_INFO: `/v3/users` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:08.003813 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc04201-31f4-49f7-bdb8-14dffc43037a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:08.005880 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc04201-31f4-49f7-bdb8-14dffc43037a admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:08.042217 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1442/2894] 10.222.0.42 () {60 vars in 1143 bytes} [Sat Jul 2 03:54:07 2022] POST /identity/v3/users => generated 297 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 02 03:54:08.043916 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443df08-3159-4816-8135-3d47819a6e3e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:08.058023 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443df08-3159-4816-8135-3d47819a6e3e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:08.065463 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231c3c3-a9e3-481c-842d-b25ec38ea0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:08.066166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231c3c3-a9e3-481c-842d-b25ec38ea0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:08.066443 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231c3c3-a9e3-481c-842d-b25ec38ea0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:08.066688 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e231c3c3-a9e3-481c-842d-b25ec38ea0c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:08.067613 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231c3c3-a9e3-481c-842d-b25ec38ea0c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:08.069657 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e231c3c3-a9e3-481c-842d-b25ec38ea0c9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:08.085591 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1453/2895] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:54:08.094438 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc04201-31f4-49f7-bdb8-14dffc43037a admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:08.105757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc04201-31f4-49f7-bdb8-14dffc43037a admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:08.112321 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332737e3-5002-40a1-8eaa-0168d6edb171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:08.113733 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332737e3-5002-40a1-8eaa-0168d6edb171 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:08.114100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332737e3-5002-40a1-8eaa-0168d6edb171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:08.114408 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332737e3-5002-40a1-8eaa-0168d6edb171 admin admin] PATH_INFO: `/v3/projects/87195cf01746437fa970ace0c04d390c/users/c0e590abd0234070a922556f384d214c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:08.133566 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332737e3-5002-40a1-8eaa-0168d6edb171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87195cf01746437fa970ace0c04d390c, user_id=c0e590abd0234070a922556f384d214c, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:08.136822 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332737e3-5002-40a1-8eaa-0168d6edb171 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:08.156311 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1443/2896] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:54:08 2022] PUT /identity/v3/projects/87195cf01746437fa970ace0c04d390c/users/c0e590abd0234070a922556f384d214c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:54:08.160562 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231c3c3-a9e3-481c-842d-b25ec38ea0c9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:08.172472 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e231c3c3-a9e3-481c-842d-b25ec38ea0c9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:08.240230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f77d26-c931-49a1-adb8-1cdea00c7aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:08.241483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f77d26-c931-49a1-adb8-1cdea00c7aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:08.241483 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f77d26-c931-49a1-adb8-1cdea00c7aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:08.241805 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f77d26-c931-49a1-adb8-1cdea00c7aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:08.242423 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f77d26-c931-49a1-adb8-1cdea00c7aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:08.244077 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f77d26-c931-49a1-adb8-1cdea00c7aa1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:08.254799 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1454/2897] 10.222.0.42 () {58 vars in 1121 bytes} [Sat Jul 2 03:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 02 03:54:08.260667 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332737e3-5002-40a1-8eaa-0168d6edb171 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:08.277368 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332737e3-5002-40a1-8eaa-0168d6edb171 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:08.283353 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1105-ed58-430b-b6fe-143738c55421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:08.284281 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1105-ed58-430b-b6fe-143738c55421 admin admin] REQUEST_METHOD: `PUT` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:08.284557 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1105-ed58-430b-b6fe-143738c55421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:08.284827 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1105-ed58-430b-b6fe-143738c55421 admin admin] PATH_INFO: `/v3/projects/87195cf01746437fa970ace0c04d390c/users/c0e590abd0234070a922556f384d214c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:08.286896 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1105-ed58-430b-b6fe-143738c55421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87195cf01746437fa970ace0c04d390c, user_id=c0e590abd0234070a922556f384d214c, role_id=d5aa55cbd9374aa2a998f4531b9f0eb0)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:08.290631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1105-ed58-430b-b6fe-143738c55421 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:08.304843 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1444/2898] 10.222.0.42 () {60 vars in 1482 bytes} [Sat Jul 2 03:54:08 2022] PUT /identity/v3/projects/87195cf01746437fa970ace0c04d390c/users/c0e590abd0234070a922556f384d214c/roles/d5aa55cbd9374aa2a998f4531b9f0eb0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 02 03:54:12.868731 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3838732-13c1-4d00-8cea-dbc54d1ba0a2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:12.869300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3838732-13c1-4d00-8cea-dbc54d1ba0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:12.869300 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3838732-13c1-4d00-8cea-dbc54d1ba0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:12.898314 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d3838732-13c1-4d00-8cea-dbc54d1ba0a2 None None] MFA Rules not processed for user `c0e590abd0234070a922556f384d214c`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:54:12.899704 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d3838732-13c1-4d00-8cea-dbc54d1ba0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:12.976430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1455/2899] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:54:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:54:12.989168 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1105-ed58-430b-b6fe-143738c55421 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:12.991097 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-50ef1105-ed58-430b-b6fe-143738c55421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:13.055422 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1105-ed58-430b-b6fe-143738c55421 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:13.059087 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075db8c3-6ae4-4580-9607-95b1eae45424 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2afd25850dea40568271538793e861c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:13.059774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db8c3-6ae4-4580-9607-95b1eae45424 service glance] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:13.060065 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db8c3-6ae4-4580-9607-95b1eae45424 service glance] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:13.060341 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db8c3-6ae4-4580-9607-95b1eae45424 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:13.087889 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:54:13.123415 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075db8c3-6ae4-4580-9607-95b1eae45424 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:13.124964 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075db8c3-6ae4-4580-9607-95b1eae45424 service glance] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:13.136080 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1445/2900] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:54:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:54:13.252497 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b38611-1417-44f2-8ceb-a86138001177 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:13.252789 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b38611-1417-44f2-8ceb-a86138001177 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:13.253067 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b38611-1417-44f2-8ceb-a86138001177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:13.270513 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:54:13.325921 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-65b38611-1417-44f2-8ceb-a86138001177 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:54:13.327159 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-65b38611-1417-44f2-8ceb-a86138001177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:13.405759 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1456/2901] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:54:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:54:13.463824 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075db8c3-6ae4-4580-9607-95b1eae45424 service glance] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:13.465924 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-075db8c3-6ae4-4580-9607-95b1eae45424 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:13.565545 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075db8c3-6ae4-4580-9607-95b1eae45424 service glance] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:13.570812 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d90cb6-5774-4e77-b026-44d665d690ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:13.571490 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d90cb6-5774-4e77-b026-44d665d690ae service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:13.571873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d90cb6-5774-4e77-b026-44d665d690ae service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:13.572227 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d90cb6-5774-4e77-b026-44d665d690ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:13.580633 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d90cb6-5774-4e77-b026-44d665d690ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:13.582258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d90cb6-5774-4e77-b026-44d665d690ae service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:13.591095 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1446/2902] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:54:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:54:23.289001 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b44b93-5c9c-485c-af6d-c7a1f9e4ffe2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:23.289840 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b44b93-5c9c-485c-af6d-c7a1f9e4ffe2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:23.290027 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b44b93-5c9c-485c-af6d-c7a1f9e4ffe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:23.310294 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-41b44b93-5c9c-485c-af6d-c7a1f9e4ffe2 None None] MFA Rules not processed for user `ed4e5bbc53a74f7083525adc968db049`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:54:23.311551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-41b44b93-5c9c-485c-af6d-c7a1f9e4ffe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:23.319561 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1457/2903] 10.222.0.42 () {58 vars in 957 bytes} [Sat Jul 2 03:54:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:54:23.588586 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d90cb6-5774-4e77-b026-44d665d690ae service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:23.589542 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=57 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:54:23.645713 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d90cb6-5774-4e77-b026-44d665d690ae service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:23.650295 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d7b2f9-512a-442f-b54c-36e373f81d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:23.651460 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7b2f9-512a-442f-b54c-36e373f81d58 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:23.651746 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7b2f9-512a-442f-b54c-36e373f81d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:23.652009 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d7b2f9-512a-442f-b54c-36e373f81d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:23.654254 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-59d7b2f9-512a-442f-b54c-36e373f81d58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:23.666238 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7b2f9-512a-442f-b54c-36e373f81d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:23.668832 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d7b2f9-512a-442f-b54c-36e373f81d58 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:23.678147 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1447/2904] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:54:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:54:26.040227 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b44b93-5c9c-485c-af6d-c7a1f9e4ffe2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:26.042389 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-41b44b93-5c9c-485c-af6d-c7a1f9e4ffe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:26.121209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b44b93-5c9c-485c-af6d-c7a1f9e4ffe2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:26.126795 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf7688-f5f7-4cc4-b06f-a6e2e136a416 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:26.127490 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf7688-f5f7-4cc4-b06f-a6e2e136a416 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:26.127774 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf7688-f5f7-4cc4-b06f-a6e2e136a416 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:26.128042 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf7688-f5f7-4cc4-b06f-a6e2e136a416 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:26.131882 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a5bf7688-f5f7-4cc4-b06f-a6e2e136a416 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:26.144719 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf7688-f5f7-4cc4-b06f-a6e2e136a416 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:26.146947 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf7688-f5f7-4cc4-b06f-a6e2e136a416 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:26.158426 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1458/2905] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:54:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:54:29.939771 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904ab59-a3ef-4785-96cf-12624d2cc8c4 None None] REQUEST_METHOD: `POST` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:29.940242 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904ab59-a3ef-4785-96cf-12624d2cc8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:29.940355 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904ab59-a3ef-4785-96cf-12624d2cc8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:29.958176 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.auth.core [None req-5904ab59-a3ef-4785-96cf-12624d2cc8c4 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80354) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:54:29.959384 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5904ab59-a3ef-4785-96cf-12624d2cc8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:29.966890 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1448/2906] 10.222.0.42 () {58 vars in 1009 bytes} [Sat Jul 2 03:54:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:54:29.979648 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf7688-f5f7-4cc4-b06f-a6e2e136a416 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:29.991209 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf7688-f5f7-4cc4-b06f-a6e2e136a416 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:29.997132 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb9046-fd23-42be-a036-98991947537b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:29.997792 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb9046-fd23-42be-a036-98991947537b service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:29.998075 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb9046-fd23-42be-a036-98991947537b service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:29.998350 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb9046-fd23-42be-a036-98991947537b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:30.007175 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb9046-fd23-42be-a036-98991947537b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:30.008801 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb9046-fd23-42be-a036-98991947537b service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:30.017980 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1459/2907] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:54:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:54:39.952979 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=29 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:54:40.013918 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=80 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:54:47.781291 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904ab59-a3ef-4785-96cf-12624d2cc8c4 None None] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:54:47.782659 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-5904ab59-a3ef-4785-96cf-12624d2cc8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:47.864732 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904ab59-a3ef-4785-96cf-12624d2cc8c4 None None] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:54:47.870258 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:54:47.870908 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:54:47.871193 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:54:47.871499 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:54:47.873304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:54:47.885573 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:54:47.887166 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:54:47.898540 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1449/2908] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:54:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:54:57.893503 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=34 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:55:00.930926 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb9046-fd23-42be-a036-98991947537b service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:55:00.933316 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9cbb9046-fd23-42be-a036-98991947537b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:55:00.953005 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb9046-fd23-42be-a036-98991947537b service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:55:00.964629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410288b5-e3da-4737-b068-cf2d9ac95497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:55:00.965537 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410288b5-e3da-4737-b068-cf2d9ac95497 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:55:00.965858 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410288b5-e3da-4737-b068-cf2d9ac95497 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:55:00.966157 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410288b5-e3da-4737-b068-cf2d9ac95497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:55:00.980413 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410288b5-e3da-4737-b068-cf2d9ac95497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:55:00.982166 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410288b5-e3da-4737-b068-cf2d9ac95497 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:55:00.995625 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1460/2909] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:55:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:55:02.849241 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:55:02.851680 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:55:02.869286 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e8eb7-4f68-4834-84c4-ef3ef24fa6b0 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:55:02.876125 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b13f30-b465-45d7-a1e6-0c2ec313f8b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:55:02.876925 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b13f30-b465-45d7-a1e6-0c2ec313f8b1 service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:55:02.877300 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b13f30-b465-45d7-a1e6-0c2ec313f8b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:55:02.877714 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b13f30-b465-45d7-a1e6-0c2ec313f8b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:55:02.879742 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-06b13f30-b465-45d7-a1e6-0c2ec313f8b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:55:02.892882 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b13f30-b465-45d7-a1e6-0c2ec313f8b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:55:02.894661 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b13f30-b465-45d7-a1e6-0c2ec313f8b1 service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:55:02.907831 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1450/2910] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:55:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:55:10.990374 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:55:12.902511 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:55:33.927015 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410288b5-e3da-4737-b068-cf2d9ac95497 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:55:33.936939 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410288b5-e3da-4737-b068-cf2d9ac95497 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:55:33.941606 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e03956-177a-459c-b941-9fa39bd25684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:55:33.942241 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e03956-177a-459c-b941-9fa39bd25684 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:55:33.942520 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e03956-177a-459c-b941-9fa39bd25684 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:55:33.942931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e03956-177a-459c-b941-9fa39bd25684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:55:33.945021 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f2e03956-177a-459c-b941-9fa39bd25684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:55:33.955144 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e03956-177a-459c-b941-9fa39bd25684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:55:33.956643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e03956-177a-459c-b941-9fa39bd25684 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:55:33.964818 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1461/2911] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:55:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:55:43.961340 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:56:05.357404 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b13f30-b465-45d7-a1e6-0c2ec313f8b1 service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:56:05.367697 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b13f30-b465-45d7-a1e6-0c2ec313f8b1 service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:56:05.372090 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb0399-a151-4db0-bf6f-afb60944c320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:56:05.372611 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb0399-a151-4db0-bf6f-afb60944c320 service neutron] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:56:05.372902 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb0399-a151-4db0-bf6f-afb60944c320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:56:05.373161 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb0399-a151-4db0-bf6f-afb60944c320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:56:05.375037 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-70fb0399-a151-4db0-bf6f-afb60944c320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:56:05.389314 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb0399-a151-4db0-bf6f-afb60944c320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:56:05.390774 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb0399-a151-4db0-bf6f-afb60944c320 service neutron] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:56:05.402466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1451/2912] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:56:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 02 03:56:06.599827 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e03956-177a-459c-b941-9fa39bd25684 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:56:06.609202 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e03956-177a-459c-b941-9fa39bd25684 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:56:06.613523 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1dd1a-f55a-4354-a44e-a6e45fbe993f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd67be26446324c91b956e9d930e04b4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:56:06.614167 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1dd1a-f55a-4354-a44e-a6e45fbe993f service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:56:06.614449 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1dd1a-f55a-4354-a44e-a6e45fbe993f service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:56:06.614720 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1dd1a-f55a-4354-a44e-a6e45fbe993f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:56:06.616551 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1ad1dd1a-f55a-4354-a44e-a6e45fbe993f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:56:06.631713 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1dd1a-f55a-4354-a44e-a6e45fbe993f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:56:06.633161 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1dd1a-f55a-4354-a44e-a6e45fbe993f service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:56:06.645130 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1462/2913] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:56:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:56:15.397412 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=6 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:56:16.639598 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:57:27.717727 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb0399-a151-4db0-bf6f-afb60944c320 service neutron] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:57:27.762670 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb0399-a151-4db0-bf6f-afb60944c320 service neutron] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:57:27.769075 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f8f7-ab30-4019-9bf8-a710bcbaf82a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b7bc20034974816905978dd365993f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:57:27.769734 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f8f7-ab30-4019-9bf8-a710bcbaf82a service nova] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:57:27.770024 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f8f7-ab30-4019-9bf8-a710bcbaf82a service nova] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:57:27.770322 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f8f7-ab30-4019-9bf8-a710bcbaf82a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:57:27.772208 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-44a9f8f7-ab30-4019-9bf8-a710bcbaf82a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:57:27.805466 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9f8f7-ab30-4019-9bf8-a710bcbaf82a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:57:27.807100 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9f8f7-ab30-4019-9bf8-a710bcbaf82a service nova] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:57:27.818700 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1452/2914] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:57:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:57:28.804777 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23110855-02b7-42db-aadb-5a7a3fab2a02 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:57:28.805301 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23110855-02b7-42db-aadb-5a7a3fab2a02 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:57:28.805451 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23110855-02b7-42db-aadb-5a7a3fab2a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:57:28.826473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-23110855-02b7-42db-aadb-5a7a3fab2a02 None None] MFA Rules not processed for user `2fcefa2d73a84415875357ef03e2c8da`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 02 03:57:28.827708 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-23110855-02b7-42db-aadb-5a7a3fab2a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:57:28.837057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1463/2915] 10.222.0.42 () {58 vars in 967 bytes} [Sat Jul 2 03:57:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 02 03:57:28.877645 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f8f7-ab30-4019-9bf8-a710bcbaf82a service nova] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:57:28.891456 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f8f7-ab30-4019-9bf8-a710bcbaf82a service nova] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:57:28.898695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa151e3-007d-4749-a2b6-a766596f39c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f37a95ba40774cfd89e6f397afa301a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:57:28.899406 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa151e3-007d-4749-a2b6-a766596f39c6 service swift] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:57:28.899895 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa151e3-007d-4749-a2b6-a766596f39c6 service swift] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:57:28.900181 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa151e3-007d-4749-a2b6-a766596f39c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:57:28.909365 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa151e3-007d-4749-a2b6-a766596f39c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:57:28.910958 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa151e3-007d-4749-a2b6-a766596f39c6 service swift] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:57:28.919937 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1453/2916] 10.222.0.42 () {60 vars in 1384 bytes} [Sat Jul 2 03:57:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 02 03:57:32.436487 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23110855-02b7-42db-aadb-5a7a3fab2a02 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:57:32.438516 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-23110855-02b7-42db-aadb-5a7a3fab2a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:57:32.455058 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23110855-02b7-42db-aadb-5a7a3fab2a02 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:57:32.460336 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f3b5e-032c-4971-9d71-864ac80815d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:57:32.460987 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f3b5e-032c-4971-9d71-864ac80815d6 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:57:32.461267 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f3b5e-032c-4971-9d71-864ac80815d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:57:32.461608 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f3b5e-032c-4971-9d71-864ac80815d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:57:32.463735 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-984f3b5e-032c-4971-9d71-864ac80815d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:57:32.476046 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f3b5e-032c-4971-9d71-864ac80815d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:57:32.477977 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f3b5e-032c-4971-9d71-864ac80815d6 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:57:32.489373 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1464/2917] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:57:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:57:35.613602 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa151e3-007d-4749-a2b6-a766596f39c6 service swift] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:57:35.626692 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa151e3-007d-4749-a2b6-a766596f39c6 service swift] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:57:35.632873 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e2b25-21c2-4f51-9719-ddf97e99d6ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:57:35.633717 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e2b25-21c2-4f51-9719-ddf97e99d6ea service placement] REQUEST_METHOD: `GET` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:57:35.634094 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e2b25-21c2-4f51-9719-ddf97e99d6ea service placement] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:57:35.634695 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e2b25-21c2-4f51-9719-ddf97e99d6ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:57:35.636817 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-aa6e2b25-21c2-4f51-9719-ddf97e99d6ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:57:35.661185 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e2b25-21c2-4f51-9719-ddf97e99d6ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:57:35.662402 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e2b25-21c2-4f51-9719-ddf97e99d6ea service placement] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:57:35.674420 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1454/2918] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:57:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:57:35.686945 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f3b5e-032c-4971-9d71-864ac80815d6 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:57:35.772230 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f3b5e-032c-4971-9d71-864ac80815d6 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:57:35.777266 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:57:35.778430 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:57:35.778703 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:57:35.778974 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] PATH_INFO: `/v3/users/ed4e5bbc53a74f7083525adc968db049` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:57:35.780245 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4e5bbc53a74f7083525adc968db049)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:57:35.782203 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:57:35.833242 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4e5bbc53a74f7083525adc968db049'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:35.833898 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4e5bbc53a74f7083525adc968db049'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:35.843057 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4e5bbc53a74f7083525adc968db049'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:35.855118 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4e5bbc53a74f7083525adc968db049'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:35.855788 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4e5bbc53a74f7083525adc968db049'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:35.867925 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1465/2919] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:57:35 2022] DELETE /identity/v3/users/ed4e5bbc53a74f7083525adc968db049 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:57:36.061409 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e2b25-21c2-4f51-9719-ddf97e99d6ea service placement] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:57:36.072646 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e2b25-21c2-4f51-9719-ddf97e99d6ea service placement] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:57:36.132757 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:57:36.133631 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:57:36.133913 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:57:36.134388 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] PATH_INFO: `/v3/projects/642a93dd7b34496491123f30dce4438d` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:57:36.135691 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=642a93dd7b34496491123f30dce4438d)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:57:36.137609 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:57:36.184745 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642a93dd7b34496491123f30dce4438d', 'request_id': 'req-05c33fad-8113-4a1b-812a-658f77fef373'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.194018 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642a93dd7b34496491123f30dce4438d', 'request_id': 'req-05c33fad-8113-4a1b-812a-658f77fef373'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.200747 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642a93dd7b34496491123f30dce4438d', 'request_id': 'req-05c33fad-8113-4a1b-812a-658f77fef373'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.211006 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:57:36.225640 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] The token cache is being invalidate because project 642a93dd7b34496491123f30dce4438d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:57:36.225951 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.226572 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.228304 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1455/2920] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:57:36 2022] DELETE /identity/v3/projects/642a93dd7b34496491123f30dce4438d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:57:36.232357 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:57:36.233869 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:57:36.247428 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbb1311-c83f-43d8-a885-d441beae3a96 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:57:36.288484 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:57:36.306911 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:57:36.307886 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:57:36.308169 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:57:36.308438 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] PATH_INFO: `/v3/users/c0e590abd0234070a922556f384d214c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:57:36.309635 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e590abd0234070a922556f384d214c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:57:36.311629 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:57:36.362008 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e590abd0234070a922556f384d214c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.362643 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e590abd0234070a922556f384d214c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.371257 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e590abd0234070a922556f384d214c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.383251 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e590abd0234070a922556f384d214c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.383890 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e590abd0234070a922556f384d214c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.397371 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1466/2921] 10.222.0.42 () {58 vars in 1223 bytes} [Sat Jul 2 03:57:36 2022] DELETE /identity/v3/users/c0e590abd0234070a922556f384d214c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:57:36.588596 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] Authenticating user token {{(pid=80354) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:57:36.590116 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.fernet_utils [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80354) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:57:36.606530 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c33fad-8113-4a1b-812a-658f77fef373 admin admin] Validating token access rules against request {{(pid=80354) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:57:36.670501 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c350860378554603affd063cfacfc453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08e65693c4ea4b43b13797fe93e3a44a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80354) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:57:36.671429 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:57:36.671712 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:57:36.671993 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] PATH_INFO: `/v3/projects/87195cf01746437fa970ace0c04d390c` {{(pid=80354) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:57:36.673198 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=87195cf01746437fa970ace0c04d390c)` {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:57:36.675131 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] RBAC: Authorization granted {{(pid=80354) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:57:36.723575 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87195cf01746437fa970ace0c04d390c', 'request_id': 'req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.733211 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87195cf01746437fa970ace0c04d390c', 'request_id': 'req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.739691 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87195cf01746437fa970ace0c04d390c', 'request_id': 'req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c'} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.750057 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] The token cache is being invalidate because project 87195cf01746437fa970ace0c04d390c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80354) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 02 03:57:36.750347 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.750977 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG keystone.notifications [None req-6909a3c6-b1f7-4a86-ab68-dd554cfe6b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80354) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 02 03:57:36.752578 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: [pid: 80354|app: 0|req: 1456/2922] 10.222.0.42 () {58 vars in 1232 bytes} [Sat Jul 2 03:57:36 2022] DELETE /identity/v3/projects/87195cf01746437fa970ace0c04d390c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 02 03:57:46.393473 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:57:46.747213 dv-cinder-846842-9-iscsi devstack@keystone.service[80354]: DEBUG dbcounter [-] [80354] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80354) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 02 03:58:30.267534 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 02 03:58:30.269345 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:58:30.363247 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8348d-c791-4ed4-9a87-ed4cd1a4ac5d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 02 03:58:30.427837 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d030db7-2f6d-437b-92d2-9f4a3f3e545d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee7acbd4c5545c1885faffb71f6306b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd9882d1720c04bd2aa92db3170e36582', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 02 03:58:30.428387 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030db7-2f6d-437b-92d2-9f4a3f3e545d service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 02 03:58:30.428658 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030db7-2f6d-437b-92d2-9f4a3f3e545d service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 02 03:58:30.428931 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d030db7-2f6d-437b-92d2-9f4a3f3e545d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 02 03:58:30.430833 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5d030db7-2f6d-437b-92d2-9f4a3f3e545d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 02 03:58:30.442332 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d030db7-2f6d-437b-92d2-9f4a3f3e545d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 02 03:58:30.443965 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d030db7-2f6d-437b-92d2-9f4a3f3e545d service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 02 03:58:30.453904 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1467/2923] 10.222.0.42 () {60 vars in 1365 bytes} [Sat Jul 2 03:58:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 02 03:58:40.449917 dv-cinder-846842-9-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=67 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}